Mailman 3 August 2006 - Python-Dev

Re: [Python-Dev] Type of range object members
by Phillip J. Eby 17 Aug '06

17 Aug '06

At 11:46 PM 8/15/2006 -0700, Neal Norwitz wrote: >On 8/15/06, "Martin v. Löwis" <martin(a)v.loewis.de> wrote: >> >>That penalty is already paid today. Much code dealing with >>ints has a type test whether it's an int or a long. If >>int and long become subtypes of each other or of some abstract >>type, performance will decrease even more because a subtype >>test is quite expensive if the object is neither int nor >>long (it has to traverse the entire base type hierarchy to >>find out its not inherited from int). > >I was playing around with a little patch to avoid that penalty. It >doesn't take any additional memory, just a handful of bits we aren't >using. :-) > >For the more common builtin types, it stores whether it's a subclass >in tp_flags, so there's no function call necessary and it's a constant >time operation. It was faster when doing simple stuff. Haven't >thought much whether this is really worthwhile or not. It seems to me that you could drop the FAST_SUBCLASS bit, since none of the other bits will be set if it is not a subclass of a builtin. That would free up one flag bit -- perhaps usable for that BaseException flag Guido wants. :) (Of course, if you can't inherit from both BaseException and one of the other builtin types, it can just be another enumeration value within the bit mask.)

3 2

Making 'python -t' the default.
by Thomas Wouters 17 Aug '06

17 Aug '06

Can I suggest making 'python -t' the default, in 2.6? It makes python warn about mixing tabs and spaces in indentation. In Py3k, '-tt' (the error-raising version) will be the default, instead. I see too many newsbies with problems they can't figure out because they mix tabs and spaces (usually not even consciously, but just because their editor isn't playing along.) There probably should be an option to turn the warning off again (although I won't fight for it ;) -- but I don't know what that option should be. -- Thomas Wouters <thomas(a)python.org> Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

4 3

no remaining issues blocking 2.5 release
by Neal Norwitz 16 Aug '06

16 Aug '06

I just updated the PEP to remove all references to issues blocking release of 2.5. I don't know of any. I haven't heard of any issues with the fixes that have been checked in. If you have issues, respond ASAP! The release candidate is planned to be cut this Thursday/Friday. There are only a few more days before code freeze. A branch will be made when the release candidate is cut. n

10 22

IDLE patches - bugfix or not?
by Jim Jewett 16 Aug '06

16 Aug '06

I just uploaded a series of IDLE patches, but I'm not quite sure how to classify them on the feature/bugfix scale now that the last beta is out. >From most to least buggish: python.org/sf/1540892 -- honor the new quit() behavior. On the other hand, it was documented that this didn't work in IDLE, and it is *possible* that someone was counting on this. python.org/sf/1540851 -- with is now a blockopener, and should be counted as such -- I *think* this one would be safe, but I know that changing a parser can be surprising, and I suppose it *could* wait until with no longer requires a future statement. python.org/sf/1540874 -- broken shortcut keys. On windows, only one entry per menu can be reached with the same shortcut letter, so advertising others is just an attractive nuisance. I'm not sure that other systems wouldn't be able to use the hidden shortcuts. python.org/sf/1540869 -- GUI fix. The current code puts in a separator using a magic number (and has XXX comments about it.) This changes the magic number so that the separator is more visible, but I'm not sure the old behavior rose to a bug, or that it wasn't platform dependent. python.org/sf/1540849 -- except too broad. I wouldn't suggest applying this late in the release cycle, except that it seems sort of like the memory errors that are still being patched. -jJ

3 3

Weekly Python Patch/Bug Summary
by Kurt B. Kaiser 16 Aug '06

16 Aug '06

Patch / Bug Summary ___________________ Patches : 404 open ( +2) / 3376 closed (+16) / 3780 total (+18) Bugs : 860 open ( -1) / 6131 closed (+17) / 6991 total (+16) RFE : 229 open ( +1) / 235 closed ( +1) / 464 total ( +2) New / Reopened Patches ______________________ option to leave tempfile.NamedTemporaryFile around on close (2006-08-10) http://python.org/sf/1537850 opened by djmdjm Patch to fix __index__() clipping (2006-08-11) CLOSED http://python.org/sf/1538606 opened by Travis Oliphant Patch cElementTree to export CurrentLineNumber (2006-08-11) http://python.org/sf/1538691 opened by Robin Bryce Replace unicode.__eq__ exceptions with a warning (2006-08-11) CLOSED http://python.org/sf/1538956 opened by M.-A. Lemburg Add readinto method to StringIO and cStringIO (2006-08-12) http://python.org/sf/1539381 opened by Alexander Belopolsky Backports from trunk to release24-maint (2006-08-15) http://python.org/sf/1540329 opened by flub tarfile __slots__ addition (2006-08-14) http://python.org/sf/1540385 opened by Brian Harring Patches for OpenBSD 4.0 (2006-08-15) http://python.org/sf/1540470 opened by djmdjm Use Py_ssize_t for rangeobject members (2006-08-15) http://python.org/sf/1540617 opened by Alexander Belopolsky except too broad (2006-08-15) http://python.org/sf/1540849 opened by Jim Jewett with is now a blockopener (2006-08-15) CLOSED http://python.org/sf/1540851 opened by Jim Jewett IOBinding overly broad except (2nd try) (2006-08-15) CLOSED http://python.org/sf/1540856 opened by Jim Jewett except too broad (2006-08-15) CLOSED http://python.org/sf/1540857 opened by Jim Jewett IOBinding broad except - try 3 (2006-08-15) CLOSED http://python.org/sf/1540859 opened by Jim Jewett CodeContext visibility (2006-08-15) http://python.org/sf/1540869 opened by Jim Jewett broken shortcut keys (2006-08-15) http://python.org/sf/1540874 opened by Jim Jewett make IDLE honor the quit() builtin (2006-08-15) CLOSED http://python.org/sf/1540892 opened by Jim Jewett Patches Closed ______________ Add notes on locale module changes to whatsnew25.tex (2006-08-03) http://python.org/sf/1534027 closed by akuchling Build ctypes on OpenBSD x86_64 (2006-08-08) http://python.org/sf/1536908 closed by theller Fix __index__() clipping really big numbers (2006-07-29) http://python.org/sf/1530738 closed by ncoghlan PyShell.recall - fix indentation logic (2006-07-25) http://python.org/sf/1528468 closed by kbk Patch to fix __index__() clipping (2006-08-11) http://python.org/sf/1538606 closed by nnorwitz Replace unicode.__eq__ exceptions with a warning (2006-08-11) http://python.org/sf/1538956 closed by lemburg Replace the ctypes internal '_as_parameter_' mechanism (2006-08-02) http://python.org/sf/1532975 closed by theller writelines() in bz2 module does not raise check for errors (2006-08-06) http://python.org/sf/1535500 closed by gbrandl trace.py on win32 has problems with lowercase drive names (2006-08-07) http://python.org/sf/1536071 closed by gbrandl Give Cookie.py its own _idmap (2006-07-13) http://python.org/sf/1521882 closed by gbrandl socket.gethostbyaddr fix for machines with incomplete DNS (2006-06-23) http://python.org/sf/1511317 closed by gbrandl with is now a blockopener (2006-08-15) http://python.org/sf/1540851 closed by kbk IOBinding overly broad except (2nd try) (2006-08-15) http://python.org/sf/1540856 closed by jimjjewett except too broad (2006-08-15) http://python.org/sf/1540857 closed by jimjjewett IOBinding broad except - try 3 (2006-08-15) http://python.org/sf/1540859 closed by jimjjewett make IDLE honor the quit() builtin (2006-08-15) http://python.org/sf/1540892 closed by kbk New / Reopened Bugs ___________________ 2nd issue with need for speed patch (2006-08-09) http://python.org/sf/1537167 opened by Robin Bryce Missing platform information in subprocess documentation (2006-08-09) CLOSED http://python.org/sf/1537195 opened by Aaron Bingham urllib2 httplib _read_chunked timeout (2006-08-09) http://python.org/sf/1537445 opened by devloop Installation on Windows Longhorn (2006-08-10) http://python.org/sf/1537601 opened by O.R.Senthil Kumaran import on cElementTree on Windows (2006-08-09) http://python.org/sf/1537685 reopened by thomasbhickey import on cElementTree on Windows (2006-08-09) http://python.org/sf/1537685 opened by Thomas B Hickey indent changes when copying command (2006-08-10) CLOSED http://python.org/sf/1538445 opened by mjd__ PyThreadState_SetAsyncExc bug (2006-08-11) http://python.org/sf/1538556 opened by ganges master pyo's are not overwritten by different optimization levels (2006-08-11) http://python.org/sf/1538778 opened by Toshio Kuratomi Long command lines don't work on Windows (2006-08-12) http://python.org/sf/1539295 opened by Albert Strasheim distutils example code missing imports (2006-08-13) CLOSED http://python.org/sf/1539336 opened by Albert Strasheim Identifiers begining with __ renamed (2006-08-14) CLOSED http://python.org/sf/1539847 opened by W Barnes warnings in interactive sessions (2006-08-14) http://python.org/sf/1539925 opened by M.-A. Lemburg Closing a pipe can lead to uncompleted command (2006-08-14) http://python.org/sf/1539954 opened by Stefan Sonnenberg Insane format string should cause ValueError (2006-08-14) CLOSED http://python.org/sf/1539955 opened by Nick Coghlan SocketServer.ForkingMixIn.collect_children() waits on pid 0 (2006-08-14) http://python.org/sf/1540386 opened by Neal Norwitz cgi.py error on parsing/handling content-disposition (2006-08-15) http://python.org/sf/1540529 opened by Dirk Holtwick Bugs Closed ___________ PyArg_ParseTupleAndKeywords potential core dump (2006-07-17) http://python.org/sf/1523610 closed by gbrandl Missing platform information in subprocess documentation (2006-08-09) http://python.org/sf/1537195 closed by gbrandl Concatenation on a long string breaks (2006-07-21) http://python.org/sf/1526585 closed by arigo import on cElementTree on Windows (2006-08-09) http://python.org/sf/1537685 closed by gbrandl cgi.FieldStorage memory usage can spike in line-oriented ops (2005-01-30) http://python.org/sf/1112549 closed by gvanrossum tokenize module does not detect inconsistent dedents (2005-06-21) http://python.org/sf/1224621 closed by gbrandl fix inplace assignment for immutable sequences (2006-02-21) http://python.org/sf/1436226 closed by gbrandl BROWSER path with capital letters (2006-07-27) http://python.org/sf/1529655 closed by sf-robot struct.pack raises TypeError where it used to convert (2006-07-28) http://python.org/sf/1530559 closed by nnorwitz indent changes when copying command (2006-08-11) http://python.org/sf/1538445 closed by kbk distutils example code missing imports (2006-08-12) http://python.org/sf/1539336 closed by nnorwitz Segmentation fault when importing expat from xml.parser (2005-07-27) http://python.org/sf/1246405 closed by sf-robot CSV regression in 2.5a1: multi-line cells (2006-04-05) http://python.org/sf/1465014 closed by sf-robot Installing documentation doesn't make it show up (2003-07-06) http://python.org/sf/766842 closed by sf-robot infinite __str__ recursion in thread causes seg fault (2003-07-31) http://python.org/sf/780714 closed by sf-robot Identifiers begining with __ renamed (2006-08-14) http://python.org/sf/1539847 closed by gbrandl Insane format string should cause ValueError (2006-08-14) http://python.org/sf/1539955 closed by gbrandl CTypes _as_parameter_ not working as documented (2006-08-03) http://python.org/sf/1533481 closed by theller PythonLauncher uses incorrect working directory (2006-07-23) http://python.org/sf/1527397 closed by sf-robot Quitter object masked (2006-05-01) http://python.org/sf/1479785 closed by kbk New / Reopened RFE __________________ csv module: add header row to DictWriter (2006-08-10) http://python.org/sf/1537721 opened by ed_abraham Allow choice of copy function in shutil.copytree (2006-08-14) http://python.org/sf/1540112 opened by Tony

1 0

TRUNK FREEZE for 2.5c1, 00:00 UTC, Thursday 17th August
by Anthony Baxter 16 Aug '06

16 Aug '06

Ok, here we go... I'm declaring the TRUNK FROZEN as of 00:00 UTC on Thursday the 17th of August. This is in about 22 hours time. At this time, I'll be cutting the release25-maint branch and 2.5c1 (and all future 2.5 releases) will be from that branch. I'll send another email once the release is done - at that point, the trunk is then available for 2.6 checkins. Anthony -- Anthony Baxter <anthony(a)interlink.com.au> It's never too late to have a happy childhood.

1 0

Elementtree and Namespaces in 2.5
by Chris S 15 Aug '06

15 Aug '06

I'm happy to see Elementtree being considered for inclusion with 2.5. However, before committing to this decision, there's an issue regarding it's namespace parsing that should be addressed. Although Elmenttree is in most respects an excellent XML parser, a huge gotcha that often makes Elementtree unsuitable for many applications lies in the way it arbitrarily renames namespaces. For example, given: <h:html xmlns:xdc="http://www.xml.com/books" xmlns:h="http://www.w3.org/HTML/1998/html4"> <h:head><h:title>Book Review</h:title></h:head> <h:body> <xdc:bookreview> <xdc:title>XML: A Primer</xdc:title> <h:table> <h:tr align="center"> <h:td>Author</h:td><h:td>Price</h:td> <h:td>Pages</h:td><h:td>Date</h:td></h:tr> <h:tr align="left"> <h:td><xdc:author>Simon St. Laurent</xdc:author></h:td> <h:td><xdc:price>31.98</xdc:price></h:td> <h:td><xdc:pages>352</xdc:pages></h:td> <h:td><xdc:date>1998/01</xdc:date></h:td> </h:tr> </h:table> </xdc:bookreview> </h:body> </h:html> Elementtree would rewrite this as: <ns0:html xmlns:ns0="http://www.w3.org/HTML/1998/html4"> <ns0:head><ns0:title>Book Review</ns0:title></ns0:head> <ns0:body> <ns1:bookreview xmlns:ns1="http://www.xml.com/books"> <ns1:title>XML: A Primer</ns1:title> <ns0:table> <ns0:tr align="center"> <ns0:td>Author</ns0:td><ns0:td>Price</ns0:td> <ns0:td>Pages</ns0:td><ns0:td>Date</ns0:td></ns0:tr> <ns0:tr align="left"> <ns0:td><ns1:author>Simon St. Laurent</ns1:author></ns0:td> <ns0:td><ns1:price>31.98</ns1:price></ns0:td> <ns0:td><ns1:pages>352</ns1:pages></ns0:td> <ns0:td><ns1:date>1998/01</ns1:date></ns0:td> </ns0:tr> </ns0:table> </ns1:bookreview> </ns0:body> </ns0:html> There's been some discussion in comp.lang.python about this functionality (http://groups.google.com/group/comp.lang.python/browse_thread/thread/31b2e9…) and while most users and the w3 spec (http://www.w3.org/TR/2001/REC-xml-c14n-20010315#NoNSPrefixRewriting) agree this feature is actually a bug, Fredrik Lundh has refused to fix this problem. Of course, this is his right. Unfortunately, Elementtree's design makes a work-around rather awkward. Therefore, we might want to rethink inclusion of Elementtree in the stdlib, or at least patch the stdlib's version of Elementtree to produce an output more in line with the w3 standard. Sincerely, Chris Spencer

4 4

Arlington VA sprint on Sept. 23
by A.M. Kuchling 15 Aug '06

15 Aug '06

The CanDo group continues to have sprints in Arlington, so we may as well continue to piggyback on their space. The next one will be Saturday Sept. 23; sign up at <http://wiki.python.org/moin/ArlingtonSprint>. If the PEP 356 plan holds, Python 2.5 will have been released a week and a half before. I plan to work on the bug/patch backlog (try searching for the Python 2.3 bugs on SF and see how many open bugs there are). --amk

5 8

Fwd: [Python-checkins] r51236 - in python/trunk: Doc/api/abstract.tex Include/abstract.h Include/object.h Lib/test/test_index.py Misc/NEWS Modules/arraymodule.c Modules/mmapmodule.c Modules/operator.c Objects/abstract.c Objects/classobject.c Objects/
by Neal Norwitz 14 Aug '06

14 Aug '06

I checked in this fix for the __index__ clipping issue that's been discussed. This patch is an improvement, but still needs some work. Please pull out any parts you have an issue with and suggest a patch to address your concern. n ---------- Forwarded message ---------- From: neal.norwitz <python-checkins(a)python.org> Date: Aug 12, 2006 10:03 AM Subject: [Python-checkins] r51236 - in python/trunk: Doc/api/abstract.tex Include/abstract.h Include/object.h Lib/test/test_index.py Misc/NEWS Modules/arraymodule.c Modules/mmapmodule.c Modules/operator.c Objects/abstract.c Objects/classobject.c Objects/intobject.c Objects/listobject.c Objects/longobject.c Objects/sliceobject.c Objects/stringobject.c Objects/tupleobject.c Objects/typeobject.c Objects/unicodeobject.c Python/ceval.c To: python-checkins(a)python.org Author: neal.norwitz Date: Sat Aug 12 19:03:09 2006 New Revision: 51236 Modified: python/trunk/Doc/api/abstract.tex python/trunk/Include/abstract.h python/trunk/Include/object.h python/trunk/Lib/test/test_index.py python/trunk/Misc/NEWS python/trunk/Modules/arraymodule.c python/trunk/Modules/mmapmodule.c python/trunk/Modules/operator.c python/trunk/Objects/abstract.c python/trunk/Objects/classobject.c python/trunk/Objects/intobject.c python/trunk/Objects/listobject.c python/trunk/Objects/longobject.c python/trunk/Objects/sliceobject.c python/trunk/Objects/stringobject.c python/trunk/Objects/tupleobject.c python/trunk/Objects/typeobject.c python/trunk/Objects/unicodeobject.c python/trunk/Python/ceval.c Log: Patch #1538606, Patch to fix __index__() clipping. I modified this patch some by fixing style, some error checking, and adding XXX comments. This patch requires review and some changes are to be expected. I'm checking in now to get the greatest possible review and establish a baseline for moving forward. I don't want this to hold up release if possible. Modified: python/trunk/Doc/api/abstract.tex ============================================================================== --- python/trunk/Doc/api/abstract.tex (original) +++ python/trunk/Doc/api/abstract.tex Sat Aug 12 19:03:09 2006 @@ -693,12 +693,31 @@ \samp{float(\var{o})}.\bifuncindex{float} \end{cfuncdesc} -\begin{cfuncdesc}{Py_ssize_t}{PyNumber_Index}{PyObject *o} - Returns the \var{o} converted to a Py_ssize_t integer on success, or - -1 with an exception raised on failure. +\begin{cfuncdesc}{PyObject*}{PyNumber_Index}{PyObject *o} + Returns the \var{o} converted to a Python int or long on success or \NULL{} + with a TypeError exception raised on failure. \versionadded{2.5} \end{cfuncdesc} +\begin{cfuncdesc}{Py_ssize_t}{PyNumber_AsSsize_t}{PyObject *o, PyObject *exc} + Returns \var{o} converted to a Py_ssize_t value if \var{o} + can be interpreted as an integer. If \var{o} can be converted to a Python + int or long but the attempt to convert to a Py_ssize_t value + would raise an \exception{OverflowError}, then the \var{exc} argument + is the type of exception that will be raised (usually \exception{IndexError} + or \exception{OverflowError}). If \var{exc} is \NULL{}, then the exception + is cleared and the value is clipped to \var{PY_SSIZE_T_MIN} + for a negative integer or \var{PY_SSIZE_T_MAX} for a positive integer. + \versionadded{2.5} +\end{cfuncdesc} + +\begin{cfuncdesc}{int}{PyIndex_Check}{PyObject *o} + Returns True if \var{o} is an index integer (has the nb_index slot of + the tp_as_number structure filled in). + \versionadded{2.5} +\end{cfuncdesc} + + \section{Sequence Protocol \label{sequence}} \begin{cfuncdesc}{int}{PySequence_Check}{PyObject *o} Modified: python/trunk/Include/abstract.h ============================================================================== --- python/trunk/Include/abstract.h (original) +++ python/trunk/Include/abstract.h Sat Aug 12 19:03:09 2006 @@ -758,13 +758,27 @@ */ - PyAPI_FUNC(Py_ssize_t) PyNumber_Index(PyObject *); +#define PyIndex_Check(obj) \ + ((obj)->ob_type->tp_as_number != NULL && \ + PyType_HasFeature((obj)->ob_type, Py_TPFLAGS_HAVE_INDEX) && \ + (obj)->ob_type->tp_as_number->nb_index != NULL) + + PyAPI_FUNC(PyObject *) PyNumber_Index(PyObject *o); /* - Returns the object converted to Py_ssize_t on success - or -1 with an error raised on failure. + Returns the object converted to a Python long or int + or NULL with an error raised on failure. */ + PyAPI_FUNC(Py_ssize_t) PyNumber_AsSsize_t(PyObject *o, PyObject *exc); + + /* + Returns the object converted to Py_ssize_t by going through + PyNumber_Index first. If an overflow error occurs while + converting the int-or-long to Py_ssize_t, then the second argument + is the error-type to return. If it is NULL, then the overflow error + is cleared and the value is clipped. + */ PyAPI_FUNC(PyObject *) PyNumber_Int(PyObject *o); Modified: python/trunk/Include/object.h ============================================================================== --- python/trunk/Include/object.h (original) +++ python/trunk/Include/object.h Sat Aug 12 19:03:09 2006 @@ -208,7 +208,7 @@ binaryfunc nb_inplace_true_divide; /* Added in release 2.5 */ - lenfunc nb_index; + unaryfunc nb_index; } PyNumberMethods; typedef struct { Modified: python/trunk/Lib/test/test_index.py ============================================================================== --- python/trunk/Lib/test/test_index.py (original) +++ python/trunk/Lib/test/test_index.py Sat Aug 12 19:03:09 2006 @@ -1,6 +1,7 @@ import unittest from test import test_support import operator +from sys import maxint class oldstyle: def __index__(self): @@ -10,68 +11,115 @@ def __index__(self): return self.ind +class TrapInt(int): + def __index__(self): + return self + +class TrapLong(long): + def __index__(self): + return self + class BaseTestCase(unittest.TestCase): def setUp(self): self.o = oldstyle() self.n = newstyle() - self.o2 = oldstyle() - self.n2 = newstyle() def test_basic(self): self.o.ind = -2 self.n.ind = 2 - assert(self.seq[self.n] == self.seq[2]) - assert(self.seq[self.o] == self.seq[-2]) - assert(operator.index(self.o) == -2) - assert(operator.index(self.n) == 2) + self.assertEqual(operator.index(self.o), -2) + self.assertEqual(operator.index(self.n), 2) + + def test_slice(self): + self.o.ind = 1 + self.n.ind = 2 + slc = slice(self.o, self.o, self.o) + check_slc = slice(1, 1, 1) + self.assertEqual(slc.indices(self.o), check_slc.indices(1)) + slc = slice(self.n, self.n, self.n) + check_slc = slice(2, 2, 2) + self.assertEqual(slc.indices(self.n), check_slc.indices(2)) + def test_wrappers(self): + self.o.ind = 4 + self.n.ind = 5 + self.assertEqual(6 .__index__(), 6) + self.assertEqual(-7L.__index__(), -7) + self.assertEqual(self.o.__index__(), 4) + self.assertEqual(self.n.__index__(), 5) + + def test_infinite_recursion(self): + self.failUnlessRaises(TypeError, operator.index, TrapInt()) + self.failUnlessRaises(TypeError, operator.index, TrapLong()) + self.failUnless(slice(TrapInt()).indices(0)==(0,0,1)) + self.failUnlessRaises(TypeError, slice(TrapLong()).indices, 0) + def test_error(self): self.o.ind = 'dumb' self.n.ind = 'bad' - myfunc = lambda x, obj: obj.seq[x] self.failUnlessRaises(TypeError, operator.index, self.o) self.failUnlessRaises(TypeError, operator.index, self.n) - self.failUnlessRaises(TypeError, myfunc, self.o, self) - self.failUnlessRaises(TypeError, myfunc, self.n, self) + self.failUnlessRaises(TypeError, slice(self.o).indices, 0) + self.failUnlessRaises(TypeError, slice(self.n).indices, 0) + + +class SeqTestCase(unittest.TestCase): + # This test case isn't run directly. It just defines common tests + # to the different sequence types below + def setUp(self): + self.o = oldstyle() + self.n = newstyle() + self.o2 = oldstyle() + self.n2 = newstyle() + + def test_index(self): + self.o.ind = -2 + self.n.ind = 2 + self.assertEqual(self.seq[self.n], self.seq[2]) + self.assertEqual(self.seq[self.o], self.seq[-2]) def test_slice(self): self.o.ind = 1 self.o2.ind = 3 self.n.ind = 2 self.n2.ind = 4 - assert(self.seq[self.o:self.o2] == self.seq[1:3]) - assert(self.seq[self.n:self.n2] == self.seq[2:4]) + self.assertEqual(self.seq[self.o:self.o2], self.seq[1:3]) + self.assertEqual(self.seq[self.n:self.n2], self.seq[2:4]) def test_repeat(self): self.o.ind = 3 self.n.ind = 2 - assert(self.seq * self.o == self.seq * 3) - assert(self.seq * self.n == self.seq * 2) - assert(self.o * self.seq == self.seq * 3) - assert(self.n * self.seq == self.seq * 2) + self.assertEqual(self.seq * self.o, self.seq * 3) + self.assertEqual(self.seq * self.n, self.seq * 2) + self.assertEqual(self.o * self.seq, self.seq * 3) + self.assertEqual(self.n * self.seq, self.seq * 2) def test_wrappers(self): - n = self.n - n.ind = 5 - assert n.__index__() == 5 - assert 6 .__index__() == 6 - assert -7L.__index__() == -7 - assert self.seq.__getitem__(n) == self.seq[5] - assert self.seq.__mul__(n) == self.seq * 5 - assert self.seq.__rmul__(n) == self.seq * 5 - - def test_infinite_recusion(self): - class Trap1(int): - def __index__(self): - return self - class Trap2(long): - def __index__(self): - return self - self.failUnlessRaises(TypeError, operator.getitem, self.seq, Trap1()) - self.failUnlessRaises(TypeError, operator.getitem, self.seq, Trap2()) + self.o.ind = 4 + self.n.ind = 5 + self.assertEqual(self.seq.__getitem__(self.o), self.seq[4]) + self.assertEqual(self.seq.__mul__(self.o), self.seq * 4) + self.assertEqual(self.seq.__rmul__(self.o), self.seq * 4) + self.assertEqual(self.seq.__getitem__(self.n), self.seq[5]) + self.assertEqual(self.seq.__mul__(self.n), self.seq * 5) + self.assertEqual(self.seq.__rmul__(self.n), self.seq * 5) + + def test_infinite_recursion(self): + self.failUnlessRaises(TypeError, operator.getitem, self.seq, TrapInt()) + self.failUnlessRaises(TypeError, operator.getitem, self.seq, TrapLong()) + + def test_error(self): + self.o.ind = 'dumb' + self.n.ind = 'bad' + indexobj = lambda x, obj: obj.seq[x] + self.failUnlessRaises(TypeError, indexobj, self.o, self) + self.failUnlessRaises(TypeError, indexobj, self.n, self) + sliceobj = lambda x, obj: obj.seq[x:] + self.failUnlessRaises(TypeError, sliceobj, self.o, self) + self.failUnlessRaises(TypeError, sliceobj, self.n, self) -class ListTestCase(BaseTestCase): +class ListTestCase(SeqTestCase): seq = [0,10,20,30,40,50] def test_setdelitem(self): @@ -82,36 +130,36 @@ del lst[self.n] lst[self.o] = 'X' lst[self.n] = 'Y' - assert lst == list('abYdefghXj') + self.assertEqual(lst, list('abYdefghXj')) lst = [5, 6, 7, 8, 9, 10, 11] lst.__setitem__(self.n, "here") - assert lst == [5, 6, "here", 8, 9, 10, 11] + self.assertEqual(lst, [5, 6, "here", 8, 9, 10, 11]) lst.__delitem__(self.n) - assert lst == [5, 6, 8, 9, 10, 11] + self.assertEqual(lst, [5, 6, 8, 9, 10, 11]) def test_inplace_repeat(self): self.o.ind = 2 self.n.ind = 3 lst = [6, 4] lst *= self.o - assert lst == [6, 4, 6, 4] + self.assertEqual(lst, [6, 4, 6, 4]) lst *= self.n - assert lst == [6, 4, 6, 4] * 3 + self.assertEqual(lst, [6, 4, 6, 4] * 3) lst = [5, 6, 7, 8, 9, 11] l2 = lst.__imul__(self.n) - assert l2 is lst - assert lst == [5, 6, 7, 8, 9, 11] * 3 + self.assert_(l2 is lst) + self.assertEqual(lst, [5, 6, 7, 8, 9, 11] * 3) -class TupleTestCase(BaseTestCase): +class TupleTestCase(SeqTestCase): seq = (0,10,20,30,40,50) -class StringTestCase(BaseTestCase): +class StringTestCase(SeqTestCase): seq = "this is a test" -class UnicodeTestCase(BaseTestCase): +class UnicodeTestCase(SeqTestCase): seq = u"this is a test" @@ -120,17 +168,47 @@ def test_xrange(self): n = newstyle() n.ind = 5 - assert xrange(1, 20)[n] == 6 - assert xrange(1, 20).__getitem__(n) == 6 + self.assertEqual(xrange(1, 20)[n], 6) + self.assertEqual(xrange(1, 20).__getitem__(n), 6) + +class OverflowTestCase(unittest.TestCase): + + def setUp(self): + self.pos = 2**100 + self.neg = -self.pos + + def test_large_longs(self): + self.assertEqual(self.pos.__index__(), self.pos) + self.assertEqual(self.neg.__index__(), self.neg) + + def test_getitem(self): + class GetItem(object): + def __len__(self): + return maxint + def __getitem__(self, key): + return key + def __getslice__(self, i, j): + return i, j + x = GetItem() + self.assertEqual(x[self.pos], self.pos) + self.assertEqual(x[self.neg], self.neg) + self.assertEqual(x[self.neg:self.pos], (-1, maxint)) + self.assertEqual(x[self.neg:self.pos:1].indices(maxint), (0, maxint, 1)) + + def test_sequence_repeat(self): + self.failUnlessRaises(OverflowError, lambda: "a" * self.pos) + self.failUnlessRaises(OverflowError, lambda: "a" * self.neg) def test_main(): test_support.run_unittest( + BaseTestCase, ListTestCase, TupleTestCase, StringTestCase, UnicodeTestCase, XRangeTestCase, + OverflowTestCase, ) if __name__ == "__main__": Modified: python/trunk/Misc/NEWS ============================================================================== --- python/trunk/Misc/NEWS (original) +++ python/trunk/Misc/NEWS Sat Aug 12 19:03:09 2006 @@ -12,6 +12,11 @@ Core and builtins ----------------- +- Patch #1538606, Fix __index__() clipping. There were some problems + discovered with the API and how integers that didn't fit into Py_ssize_t + were handled. This patch attempts to provide enough alternatives + to effectively use __index__. + - Bug #1536021: __hash__ may now return long int; the final hash value is obtained by invoking hash on the long int. Modified: python/trunk/Modules/arraymodule.c ============================================================================== --- python/trunk/Modules/arraymodule.c (original) +++ python/trunk/Modules/arraymodule.c Sat Aug 12 19:03:09 2006 @@ -1572,14 +1572,11 @@ return s; } -#define HASINDEX(o) PyType_HasFeature((o)->ob_type, Py_TPFLAGS_HAVE_INDEX) - static PyObject* array_subscr(arrayobject* self, PyObject* item) { - PyNumberMethods *nb = item->ob_type->tp_as_number; - if (nb != NULL && HASINDEX(item) && nb->nb_index != NULL) { - Py_ssize_t i = nb->nb_index(item); + if (PyIndex_Check(item)) { + Py_ssize_t i = PyNumber_AsSsize_t(item, PyExc_IndexError); if (i==-1 && PyErr_Occurred()) { return NULL; } @@ -1627,9 +1624,8 @@ static int array_ass_subscr(arrayobject* self, PyObject* item, PyObject* value) { - PyNumberMethods *nb = item->ob_type->tp_as_number; - if (nb != NULL && HASINDEX(item) && nb->nb_index != NULL) { - Py_ssize_t i = nb->nb_index(item); + if (PyIndex_Check(item)) { + Py_ssize_t i = PyNumber_AsSsize_t(item, PyExc_IndexError); if (i==-1 && PyErr_Occurred()) return -1; if (i < 0) Modified: python/trunk/Modules/mmapmodule.c ============================================================================== --- python/trunk/Modules/mmapmodule.c (original) +++ python/trunk/Modules/mmapmodule.c Sat Aug 12 19:03:09 2006 @@ -808,8 +808,6 @@ }; -#define HASINDEX(o) PyType_HasFeature((o)->ob_type, Py_TPFLAGS_HAVE_INDEX) - /* extract the map size from the given PyObject Returns -1 on error, with an appropriate Python exception raised. On @@ -817,31 +815,19 @@ static Py_ssize_t _GetMapSize(PyObject *o) { - PyNumberMethods *nb = o->ob_type->tp_as_number; - if (nb != NULL && HASINDEX(o) && nb->nb_index != NULL) { - Py_ssize_t i = nb->nb_index(o); + if (PyIndex_Check(o)) { + Py_ssize_t i = PyNumber_AsSsize_t(o, PyExc_OverflowError); if (i==-1 && PyErr_Occurred()) return -1; - if (i < 0) - goto onnegoverflow; - if (i==PY_SSIZE_T_MAX) - goto onposoverflow; + if (i < 0) { + PyErr_SetString(PyExc_OverflowError, + "memory mapped size must be positive"); + return -1; + } return i; } - else { - PyErr_SetString(PyExc_TypeError, - "map size must be an integral value"); - return -1; - } - - onnegoverflow: - PyErr_SetString(PyExc_OverflowError, - "memory mapped size must be positive"); - return -1; - onposoverflow: - PyErr_SetString(PyExc_OverflowError, - "memory mapped size is too large (limited by C int)"); + PyErr_SetString(PyExc_TypeError, "map size must be an integral value"); return -1; } Modified: python/trunk/Modules/operator.c ============================================================================== --- python/trunk/Modules/operator.c (original) +++ python/trunk/Modules/operator.c Sat Aug 12 19:03:09 2006 @@ -139,15 +139,7 @@ static PyObject * op_index(PyObject *s, PyObject *a) { - Py_ssize_t i; - PyObject *a1; - if (!PyArg_UnpackTuple(a,"index", 1, 1, &a1)) - return NULL; - i = PyNumber_Index(a1); - if (i == -1 && PyErr_Occurred()) - return NULL; - else - return PyInt_FromSsize_t(i); + return PyNumber_Index(a); } static PyObject* @@ -249,7 +241,7 @@ spam1(is_, "is_(a, b) -- Same as a is b.") spam1(is_not, "is_not(a, b) -- Same as a is not b.") -spam2(index, __index__, "index(a) -- Same as a.__index__()") +spam2o(index, __index__, "index(a) -- Same as a.__index__()") spam2(add,__add__, "add(a, b) -- Same as a + b.") spam2(sub,__sub__, "sub(a, b) -- Same as a - b.") spam2(mul,__mul__, "mul(a, b) -- Same as a * b.") Modified: python/trunk/Objects/abstract.c ============================================================================== --- python/trunk/Objects/abstract.c (original) +++ python/trunk/Objects/abstract.c Sat Aug 12 19:03:09 2006 @@ -8,8 +8,6 @@ #define NEW_STYLE_NUMBER(o) PyType_HasFeature((o)->ob_type, \ Py_TPFLAGS_CHECKTYPES) -#define HASINDEX(o) PyType_HasFeature((o)->ob_type, Py_TPFLAGS_HAVE_INDEX) - /* Shorthands to return certain errors */ @@ -122,9 +120,9 @@ return m->mp_subscript(o, key); if (o->ob_type->tp_as_sequence) { - PyNumberMethods *nb = key->ob_type->tp_as_number; - if (nb != NULL && HASINDEX(key) && nb->nb_index != NULL) { - Py_ssize_t key_value = nb->nb_index(key); + if (PyIndex_Check(key)) { + Py_ssize_t key_value; + key_value = PyNumber_AsSsize_t(key, PyExc_IndexError); if (key_value == -1 && PyErr_Occurred()) return NULL; return PySequence_GetItem(o, key_value); @@ -151,9 +149,9 @@ return m->mp_ass_subscript(o, key, value); if (o->ob_type->tp_as_sequence) { - PyNumberMethods *nb = key->ob_type->tp_as_number; - if (nb != NULL && HASINDEX(key) && nb->nb_index != NULL) { - Py_ssize_t key_value = nb->nb_index(key); + if (PyIndex_Check(key)) { + Py_ssize_t key_value; + key_value = PyNumber_AsSsize_t(key, PyExc_IndexError); if (key_value == -1 && PyErr_Occurred()) return -1; return PySequence_SetItem(o, key_value, value); @@ -183,9 +181,9 @@ return m->mp_ass_subscript(o, key, (PyObject*)NULL); if (o->ob_type->tp_as_sequence) { - PyNumberMethods *nb = key->ob_type->tp_as_number; - if (nb != NULL && HASINDEX(key) && nb->nb_index != NULL) { - Py_ssize_t key_value = nb->nb_index(key); + if (PyIndex_Check(key)) { + Py_ssize_t key_value; + key_value = PyNumber_AsSsize_t(key, PyExc_IndexError); if (key_value == -1 && PyErr_Occurred()) return -1; return PySequence_DelItem(o, key_value); @@ -653,9 +651,8 @@ sequence_repeat(ssizeargfunc repeatfunc, PyObject *seq, PyObject *n) { Py_ssize_t count; - PyNumberMethods *nb = n->ob_type->tp_as_number; - if (nb != NULL && HASINDEX(n) && nb->nb_index != NULL) { - count = nb->nb_index(n); + if (PyIndex_Check(n)) { + count = PyNumber_AsSsize_t(n, PyExc_OverflowError); if (count == -1 && PyErr_Occurred()) return NULL; } @@ -938,23 +935,89 @@ return x; } -/* Return a Py_ssize_t integer from the object item */ -Py_ssize_t +/* Return a Python Int or Long from the object item + Raise TypeError if the result is not an int-or-long + or if the object cannot be interpreted as an index. +*/ +PyObject * PyNumber_Index(PyObject *item) { - Py_ssize_t value = -1; - PyNumberMethods *nb = item->ob_type->tp_as_number; - if (nb != NULL && HASINDEX(item) && nb->nb_index != NULL) { - value = nb->nb_index(item); + PyObject *result = NULL; + if (item == NULL) + return null_error(); + /* XXX(nnorwitz): should these be CheckExact? Aren't subclasses ok? */ + if (PyInt_CheckExact(item) || PyLong_CheckExact(item)) { + Py_INCREF(item); + return item; + } + if (PyIndex_Check(item)) { + result = item->ob_type->tp_as_number->nb_index(item); + /* XXX(nnorwitz): Aren't subclasses ok here too? */ + if (result && + !PyInt_CheckExact(result) && !PyLong_CheckExact(result)) { + PyErr_Format(PyExc_TypeError, + "__index__ returned non-(int,long) " \ + "(type %.200s)", + result->ob_type->tp_name); + Py_DECREF(result); + return NULL; + } } else { PyErr_Format(PyExc_TypeError, "'%.200s' object cannot be interpreted " "as an index", item->ob_type->tp_name); } - return value; + return result; } +/* Return an error on Overflow only if err is not NULL*/ + +Py_ssize_t +PyNumber_AsSsize_t(PyObject *item, PyObject *err) +{ + Py_ssize_t result; + PyObject *runerr; + PyObject *value = PyNumber_Index(item); + if (value == NULL) + return -1; + + /* We're done if PyInt_AsSsize_t() returns without error. */ + result = PyInt_AsSsize_t(value); + if (result != -1 || !(runerr = PyErr_Occurred())) + goto finish; + + /* Error handling code -- only manage OverflowError differently */ + if (!PyErr_GivenExceptionMatches(runerr, PyExc_OverflowError)) + goto finish; + + PyErr_Clear(); + /* If no error-handling desired then the default clipping + is sufficient. + */ + if (!err) { + assert(PyLong_Check(value)); + /* Whether or not it is less than or equal to + zero is determined by the sign of ob_size + */ + if (_PyLong_Sign(value) < 0) + result = PY_SSIZE_T_MIN; + else + result = PY_SSIZE_T_MAX; + } + else { + /* Otherwise replace the error with caller's error object. */ + PyErr_Format(err, + "cannot fit '%.200s' into an index-sized integer", + item->ob_type->tp_name); + } + + finish: + Py_DECREF(value); + return result; +} + + PyObject * PyNumber_Int(PyObject *o) { Modified: python/trunk/Objects/classobject.c ============================================================================== --- python/trunk/Objects/classobject.c (original) +++ python/trunk/Objects/classobject.c Sat Aug 12 19:03:09 2006 @@ -1670,40 +1670,28 @@ return outcome > 0; } -static Py_ssize_t +static PyObject * instance_index(PyInstanceObject *self) { PyObject *func, *res; - Py_ssize_t outcome; static PyObject *indexstr = NULL; if (indexstr == NULL) { indexstr = PyString_InternFromString("__index__"); if (indexstr == NULL) - return -1; + return NULL; } if ((func = instance_getattr(self, indexstr)) == NULL) { if (!PyErr_ExceptionMatches(PyExc_AttributeError)) - return -1; + return NULL; PyErr_Clear(); PyErr_SetString(PyExc_TypeError, "object cannot be interpreted as an index"); - return -1; + return NULL; } res = PyEval_CallObject(func, (PyObject *)NULL); Py_DECREF(func); - if (res == NULL) - return -1; - if (PyInt_Check(res) || PyLong_Check(res)) { - outcome = res->ob_type->tp_as_number->nb_index(res); - } - else { - PyErr_SetString(PyExc_TypeError, - "__index__ must return an int or a long"); - outcome = -1; - } - Py_DECREF(res); - return outcome; + return res; } @@ -2026,7 +2014,7 @@ instance_truediv, /* nb_true_divide */ instance_ifloordiv, /* nb_inplace_floor_divide */ instance_itruediv, /* nb_inplace_true_divide */ - (lenfunc)instance_index, /* nb_index */ + (unaryfunc)instance_index, /* nb_index */ }; PyTypeObject PyInstance_Type = { Modified: python/trunk/Objects/intobject.c ============================================================================== --- python/trunk/Objects/intobject.c (original) +++ python/trunk/Objects/intobject.c Sat Aug 12 19:03:09 2006 @@ -193,16 +193,21 @@ PyIntObject *io; Py_ssize_t val; #endif - if (op && !PyInt_CheckExact(op) && PyLong_Check(op)) + + if (op == NULL) { + PyErr_SetString(PyExc_TypeError, "an integer is required"); + return -1; + } + + if (PyInt_Check(op)) + return PyInt_AS_LONG((PyIntObject*) op); + if (PyLong_Check(op)) return _PyLong_AsSsize_t(op); #if SIZEOF_SIZE_T == SIZEOF_LONG return PyInt_AsLong(op); #else - if (op && PyInt_Check(op)) - return PyInt_AS_LONG((PyIntObject*) op); - - if (op == NULL || (nb = op->ob_type->tp_as_number) == NULL || + if ((nb = op->ob_type->tp_as_number) == NULL || (nb->nb_int == NULL && nb->nb_long == 0)) { PyErr_SetString(PyExc_TypeError, "an integer is required"); return -1; @@ -1079,7 +1084,7 @@ int_true_divide, /* nb_true_divide */ 0, /* nb_inplace_floor_divide */ 0, /* nb_inplace_true_divide */ - PyInt_AsSsize_t, /* nb_index */ + (unaryfunc)int_int, /* nb_index */ }; PyTypeObject PyInt_Type = { Modified: python/trunk/Objects/listobject.c ============================================================================== --- python/trunk/Objects/listobject.c (original) +++ python/trunk/Objects/listobject.c Sat Aug 12 19:03:09 2006 @@ -2450,14 +2450,13 @@ "list() -> new list\n" "list(sequence) -> new list initialized from sequence's items"); -#define HASINDEX(o) PyType_HasFeature((o)->ob_type, Py_TPFLAGS_HAVE_INDEX) static PyObject * list_subscript(PyListObject* self, PyObject* item) { - PyNumberMethods *nb = item->ob_type->tp_as_number; - if (nb != NULL && HASINDEX(item) && nb->nb_index != NULL) { - Py_ssize_t i = nb->nb_index(item); + if (PyIndex_Check(item)) { + Py_ssize_t i; + i = PyNumber_AsSsize_t(item, PyExc_IndexError); if (i == -1 && PyErr_Occurred()) return NULL; if (i < 0) @@ -2504,9 +2503,8 @@ static int list_ass_subscript(PyListObject* self, PyObject* item, PyObject* value) { - PyNumberMethods *nb = item->ob_type->tp_as_number; - if (nb != NULL && HASINDEX(item) && nb->nb_index != NULL) { - Py_ssize_t i = nb->nb_index(item); + if (PyIndex_Check(item)) { + Py_ssize_t i = PyNumber_AsSsize_t(item, PyExc_IndexError); if (i == -1 && PyErr_Occurred()) return -1; if (i < 0) Modified: python/trunk/Objects/longobject.c ============================================================================== --- python/trunk/Objects/longobject.c (original) +++ python/trunk/Objects/longobject.c Sat Aug 12 19:03:09 2006 @@ -240,8 +240,11 @@ return -1; } -static Py_ssize_t -_long_as_ssize_t(PyObject *vv) { +/* Get a Py_ssize_t from a long int object. + Returns -1 and sets an error condition if overflow occurs. */ + +Py_ssize_t +_PyLong_AsSsize_t(PyObject *vv) { register PyLongObject *v; size_t x, prev; Py_ssize_t i; @@ -277,45 +280,7 @@ overflow: PyErr_SetString(PyExc_OverflowError, "long int too large to convert to int"); - if (sign > 0) - return PY_SSIZE_T_MAX; - else - return PY_SSIZE_T_MIN; -} - -/* Get a Py_ssize_t from a long int object. - Returns -1 and sets an error condition if overflow occurs. */ - -Py_ssize_t -_PyLong_AsSsize_t(PyObject *vv) -{ - Py_ssize_t x; - - x = _long_as_ssize_t(vv); - if (PyErr_Occurred()) return -1; - return x; -} - - -/* Get a Py_ssize_t from a long int object. - Silently reduce values larger than PY_SSIZE_T_MAX to PY_SSIZE_T_MAX, - and silently boost values less than -PY_SSIZE_T_MAX-1 to -PY_SSIZE_T_MAX-1. - On error, return -1 with an exception set. -*/ - -static Py_ssize_t -long_index(PyObject *vv) -{ - Py_ssize_t x; - - x = _long_as_ssize_t(vv); - if (PyErr_Occurred()) { - /* If overflow error, ignore the error */ - if (x != -1) { - PyErr_Clear(); - } - } - return x; + return -1; } /* Get a C unsigned long int from a long int object. @@ -3405,7 +3370,7 @@ long_true_divide, /* nb_true_divide */ 0, /* nb_inplace_floor_divide */ 0, /* nb_inplace_true_divide */ - long_index, /* nb_index */ + long_long, /* nb_index */ }; PyTypeObject PyLong_Type = { Modified: python/trunk/Objects/sliceobject.c ============================================================================== --- python/trunk/Objects/sliceobject.c (original) +++ python/trunk/Objects/sliceobject.c Sat Aug 12 19:03:09 2006 @@ -252,7 +252,7 @@ { Py_ssize_t ilen, start, stop, step, slicelength; - ilen = PyInt_AsSsize_t(len); + ilen = PyNumber_AsSsize_t(len, PyExc_OverflowError); if (ilen == -1 && PyErr_Occurred()) { return NULL; Modified: python/trunk/Objects/stringobject.c ============================================================================== --- python/trunk/Objects/stringobject.c (original) +++ python/trunk/Objects/stringobject.c Sat Aug 12 19:03:09 2006 @@ -1184,14 +1184,11 @@ return x; } -#define HASINDEX(o) PyType_HasFeature((o)->ob_type, Py_TPFLAGS_HAVE_INDEX) - static PyObject* string_subscript(PyStringObject* self, PyObject* item) { - PyNumberMethods *nb = item->ob_type->tp_as_number; - if (nb != NULL && HASINDEX(item) && nb->nb_index != NULL) { - Py_ssize_t i = nb->nb_index(item); + if (PyIndex_Check(item)) { + Py_ssize_t i = PyNumber_AsSsize_t(item, PyExc_IndexError); if (i == -1 && PyErr_Occurred()) return NULL; if (i < 0) Modified: python/trunk/Objects/tupleobject.c ============================================================================== --- python/trunk/Objects/tupleobject.c (original) +++ python/trunk/Objects/tupleobject.c Sat Aug 12 19:03:09 2006 @@ -577,14 +577,11 @@ (objobjproc)tuplecontains, /* sq_contains */ }; -#define HASINDEX(o) PyType_HasFeature((o)->ob_type, Py_TPFLAGS_HAVE_INDEX) - static PyObject* tuplesubscript(PyTupleObject* self, PyObject* item) { - PyNumberMethods *nb = item->ob_type->tp_as_number; - if (nb != NULL && HASINDEX(item) && nb->nb_index != NULL) { - Py_ssize_t i = nb->nb_index(item); + if (PyIndex_Check(item)) { + Py_ssize_t i = PyNumber_AsSsize_t(item, PyExc_IndexError); if (i == -1 && PyErr_Occurred()) return NULL; if (i < 0) Modified: python/trunk/Objects/typeobject.c ============================================================================== --- python/trunk/Objects/typeobject.c (original) +++ python/trunk/Objects/typeobject.c Sat Aug 12 19:03:09 2006 @@ -3527,7 +3527,7 @@ if (!PyArg_UnpackTuple(args, "", 1, 1, &o)) return NULL; - i = PyNumber_Index(o); + i = PyNumber_AsSsize_t(o, PyExc_OverflowError); if (i == -1 && PyErr_Occurred()) return NULL; return (*func)(self, i); @@ -3538,7 +3538,7 @@ { Py_ssize_t i; - i = PyNumber_Index(arg); + i = PyNumber_AsSsize_t(arg, PyExc_OverflowError); if (i == -1 && PyErr_Occurred()) return -1; if (i < 0) { @@ -4344,26 +4344,11 @@ } -static Py_ssize_t +static PyObject * slot_nb_index(PyObject *self) { static PyObject *index_str; - PyObject *temp = call_method(self, "__index__", &index_str, "()"); - Py_ssize_t result; - - if (temp == NULL) - return -1; - if (PyInt_CheckExact(temp) || PyLong_CheckExact(temp)) { - result = temp->ob_type->tp_as_number->nb_index(temp); - } - else { - PyErr_Format(PyExc_TypeError, - "__index__ must return an int or a long, " - "not '%.200s'", temp->ob_type->tp_name); - result = -1; - } - Py_DECREF(temp); - return result; + return call_method(self, "__index__", &index_str, "()"); } @@ -5109,7 +5094,7 @@ "oct(x)"), UNSLOT("__hex__", nb_hex, slot_nb_hex, wrap_unaryfunc, "hex(x)"), - NBSLOT("__index__", nb_index, slot_nb_index, wrap_lenfunc, + NBSLOT("__index__", nb_index, slot_nb_index, wrap_unaryfunc, "x[y:z] <==> x[y.__index__():z.__index__()]"), IBSLOT("__iadd__", nb_inplace_add, slot_nb_inplace_add, wrap_binaryfunc, "+"), Modified: python/trunk/Objects/unicodeobject.c ============================================================================== --- python/trunk/Objects/unicodeobject.c (original) +++ python/trunk/Objects/unicodeobject.c Sat Aug 12 19:03:09 2006 @@ -6985,14 +6985,11 @@ PyUnicode_Contains, /* sq_contains */ }; -#define HASINDEX(o) PyType_HasFeature((o)->ob_type, Py_TPFLAGS_HAVE_INDEX) - static PyObject* unicode_subscript(PyUnicodeObject* self, PyObject* item) { - PyNumberMethods *nb = item->ob_type->tp_as_number; - if (nb != NULL && HASINDEX(item) && nb->nb_index != NULL) { - Py_ssize_t i = nb->nb_index(item); + if (PyIndex_Check(item)) { + Py_ssize_t i = PyNumber_AsSsize_t(item, PyExc_IndexError); if (i == -1 && PyErr_Occurred()) return NULL; if (i < 0) Modified: python/trunk/Python/ceval.c ============================================================================== --- python/trunk/Python/ceval.c (original) +++ python/trunk/Python/ceval.c Sat Aug 12 19:03:09 2006 @@ -3866,12 +3866,14 @@ if (v != NULL) { Py_ssize_t x; if (PyInt_Check(v)) { - x = PyInt_AsSsize_t(v); + /* XXX(nnorwitz): I think PyInt_AS_LONG is correct, + however, it looks like it should be AsSsize_t. + There should be a comment here explaining why. + */ + x = PyInt_AS_LONG(v); } - else if (v->ob_type->tp_as_number && - PyType_HasFeature(v->ob_type, Py_TPFLAGS_HAVE_INDEX) - && v->ob_type->tp_as_number->nb_index) { - x = v->ob_type->tp_as_number->nb_index(v); + else if (PyIndex_Check(v)) { + x = PyNumber_AsSsize_t(v, NULL); if (x == -1 && PyErr_Occurred()) return 0; } @@ -3887,10 +3889,8 @@ } #undef ISINDEX -#define ISINDEX(x) ((x) == NULL || PyInt_Check(x) || PyLong_Check(x) || \ - ((x)->ob_type->tp_as_number && \ - PyType_HasFeature((x)->ob_type, Py_TPFLAGS_HAVE_INDEX) \ - && (x)->ob_type->tp_as_number->nb_index)) +#define ISINDEX(x) ((x) == NULL || \ + PyInt_Check(x) || PyLong_Check(x) || PyIndex_Check(x)) static PyObject * apply_slice(PyObject *u, PyObject *v, PyObject *w) /* return u[v:w] */ _______________________________________________ Python-checkins mailing list Python-checkins(a)python.org http://mail.python.org/mailman/listinfo/python-checkins

3 4

ANN: byteplay - a bytecode assembler/disassembler
by Noam Raphael 14 Aug '06

14 Aug '06

Hello, I wanted to tell you that I wrote a Python bytecode assembler/disassembler, and would be happy if people tried it and said what they think. I send this message to this list because this module deals with pretty low-level Python, so I thought it might interest the audience here. If I was wrong - please forgive me. The idea is to define an object which is equivalent to Python's code object, but which is easy to work with. To explain what I mean, I'll show a quick example. We can define this stupid function: >>> def f(a, b): ... print (a, b) We can convert it to an equivalent object, and see how it stores the byte code: >>> from byteplay import * >>> c = Code.from_code(f.func_code) >>> from pprint import pprint; pprint(c.code) [(SetLineno, 2), (LOAD_FAST, 'a'), (LOAD_FAST, 'b'), (BUILD_TUPLE, 2), (PRINT_ITEM, None), (PRINT_NEWLINE, None), (LOAD_CONST, None), (RETURN_VALUE, None)] We can change the bytecode easily, and see what happens: >>> c.code[3:3] = [(ROT_TWO, None)] >>> f.func_code = c.to_code() >>> f(3, 5) (5, 3) The idea is basically the same as that of Michael Hudson's bytecodehacks, but this one works with Python 2.4 and 2.5. I also think that it's simpler to use. I borrowed some code from Phillip J. Eby's peak.util.assembler - the main difference between his package and mine is that mine lets you play with existing bytecode, not only create new code objects. I learned a lot about Python's bytecode from writing this, and I think that other may learn from it as well - I think it's much easier to understand how code objects work by understanding equivalent objects which were meant to be as simple as possible, instead of as fast as possible. I think it got pretty good testing - I patched __import__ so that after a module is imported, all function objects (found by the gc module) were disassembled and assembled again. I then managed to get the complete test suite to pass! You can download the module from http://byteplay.googlecode.com/svn/trunk/byteplay.py . I wrote a documentation (which goes into some length in purpose of explaining how bytecode works). It's on the wiki: http://wiki.python.org/moin/ByteplayDoc . I even thought that it might get to the standard library, because it seemed to me to be a pretty good documentation of bytecode details, and because it has a pretty straightforward interface. It also makes meddling with bytecode much less dangerous - for example, you can see Raymond's original recipe for binding constants at compile time (I reposted it at http://python.pastebin.com/768312) and how using byteplay makes it simple (posted at http://python.pastebin.com/768318 so you can view the diff). But, of course - it's up to you. I will be entirely satisfied if people simply find it useful. Have a good day, Noam

1 0