Mailman 3 May 2007 - Python-Dev

PyInt_AsSsize_t on x64
by Kristján Valur Jónsson 05 May '07

05 May '07

Hello there. I'm working on getting the 64 bit build of the trunk pass the testsuite. Here is one snag, that you could help me fix. In test_getargs2(), there is at line 190: self.failUnlessEqual(99, getargs_n(Long())) Now, the Long class has a __int__ method which returns 99L. However, we run into trouble here: intobject.c:210 if ((nb = op->ob_type->tp_as_number) == NULL || (nb->nb_int == NULL && nb->nb_long == 0)) { PyErr_SetString(PyExc_TypeError, "an integer is required"); return -1; } if (nb->nb_long != 0) { io = (PyIntObject*) (*nb->nb_long) (op); } else { io = (PyIntObject*) (*nb->nb_int) (op); } trouble here The trouble is that nb->nb_long is non zero, but when called, it returns an attribute error, since __long__ is missing. nb_long points to instance_long. Now, how to fix this? Should the code in intobject.c catch the AttributeError, maybe, and continue to the nb_int? Cheers, Kristján

2 2

x64 and the testsuite
by Kristján Valur Jónsson 05 May '07

05 May '07

Hello again. A lot of overflow tests fail in the testsuite, by expecting overflow using sys.maxint. for example this line, 196, in test_index.py: self.assertEqual(x[self.neg:self.pos], (-1, maxint)) At the moment, I am disabling these tests with if "64 bit" not in sys.version: So, two questions: Should we add something like sys.maxsize to keep these overflow tests valid? Also, some tests just kill the computer when given large values, that are expected to overflow. Sometimes it would be good to test for a 64 bit machine with virtually infinite ram. Is there a better way than the "64 bit" in sys.version test? Should we have something like sys.bits? Kristján

5 5

updated PEP3125, Remove Backslash Continuation
by Jim Jewett 04 May '07

04 May '07

Major rewrite. The inside-a-string continuation is separated from the general continuation. The alternatives section is expaned to als list Andrew Koenig's improved inside-expressions variant, since that is a real contender. If anyone feels I haven't acknowledged their concerns, please tell me. -------------- PEP: 3125 Title: Remove Backslash Continuation Version: $Revision$ Last-Modified: $Date$ Author: Jim J. Jewett <JimJJewett(a)gmail.com> Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 29-Apr-2007 Post-History: 29-Apr-2007, 30-Apr-2007, 04-May-2007 Abstract ======== Python initially inherited its parsing from C. While this has been generally useful, there are some remnants which have been less useful for python, and should be eliminated. This PEP proposes elimination of terminal ``\`` as a marker for line continuation. Motivation ========== One goal for Python 3000 should be to simplify the language by removing unnecessary or duplicated features. There are currently several ways to indicate that a logical line is continued on the following physical line. The other continuation methods are easily explained as a logical consequence of the semantics they provide; ``\`` is simply an escape character that needs to be memorized. Existing Line Continuation Methods ================================== Parenthetical Expression - ([{}]) --------------------------------- Open a parenthetical expression. It doesn't matter whether people view the "line" as continuing; they do immediately recognize that the expression needs to be closed before the statement can end. An examples using each of (), [], and {}:: def fn(long_argname1, long_argname2): settings = {"background": "random noise" "volume": "barely audible"} restrictions = ["Warrantee void if used", "Notice must be recieved by yesterday" "Not responsible for sales pitch"] Note that it is always possible to parenthesize an expression, but it can seem odd to parenthesize an expression that needs them only for the line break:: assert val>4, ( "val is too small") Triple-Quoted Strings --------------------- Open a triple-quoted string; again, people recognize that the string needs to finish before the next statement starts. banner_message = """ Satisfaction Guaranteed, or DOUBLE YOUR MONEY BACK!!! some minor restrictions apply""" Terminal ``\`` in the general case ---------------------------------- A terminal ``\`` indicates that the logical line is continued on the following physical line (after whitespace). There are no particular semantics associated with this. This form is never required, although it may look better (particularly for people with a C language background) in some cases:: >>> assert val>4, \ "val is too small" Also note that the ``\`` must be the final character in the line. If your editor navigation can add whitespace to the end of a line, that invisible change will alter the semantics of the program. Fortunately, the typical result is only a syntax error, rather than a runtime bug:: >>> assert val>4, \ "val is too small" SyntaxError: unexpected character after line continuation character This PEP proposes to eliminate this redundant and potentially confusing alternative. Terminal ``\`` within a string ------------------------------ A terminal ``\`` within a single-quoted string, at the end of the line. This is arguably a special case of the terminal ``\``, but it is a special case that may be worth keeping. >>> "abd\ def" 'abd def' + Many of the objections to removing ``\`` termination were really just objections to removing it within literal strings; several people clarified that they want to keep this literal-string usage, but don't mind losing the general case. + The use of ``\`` for an escape character within strings is well known. - But note that this particular usage is odd, because the escaped character (the newline) is invisible, and the special treatment is to delete the character. That said, the ``\`` of ``\(newline)`` is still an escape which changes the meaning of the following character. Alternate Proposals =================== Several people have suggested alternative ways of marking the line end. Most of these were rejected for not actually simplifying things. The one exception was to let any unfished expression signify a line continuation, possibly in conjunction with increased indentation. This is attractive because it is a generalization of the rule for parentheses. The initial objections to this were: - The amount of whitespace may be contentious; expression continuation should not be confused with opening a new suite. - The "expression continuation" markers are not as clearly marked in Python as the grouping punctuation "(), [], {}" marks are:: # Plus needs another operand, so the line continues "abc" + "def" # String ends an expression, so the line does not # not continue. The next line is a syntax error because # unary plus does not apply to strings. "abc" + "def" - Guido objected for technical reasons. [#dedent]_ The most obvious implementation would require allowing INDENT or DEDENT tokens anywhere, or at least in a widely expanded (and ill-defined) set of locations. While this is concern only for the internal parsing mechanism (rather than for users), it would be a major new source of complexity. Andrew Koenig then pointed out [#lexical]_ a better implementation strategy, and said that it had worked quite well in other languages. [#snocone]_ The improved suggestion boiled down to:: The whitespace that follows an (operator or) open bracket or parenthesis can include newline characters. It would be implemented at a very low lexical level -- even before the decision is made to turn a newline followed by spaces into an INDENT or DEDENT token. There is still some concern that it could mask bugs, as in this example [#guidobughide]_:: # Used to be y+1, the 1 got dropped. Syntax Error (today) # would become nonsense. x = y+ f(x) Requiring that the continuation be indented more than the initial line would add both safety and complexity. Open Issues =========== + Should ``\``-continuation be removed even inside strings? + Should the continuation markers be expanced from just ([{}]) to include lines ending with an operator? + As a safety measure, should the continuation line be required to be more indented than the initial line? References ========== .. [#dedent] (email subject) PEP 30XZ: Simplified Parsing, van Rossum http://mail.python.org/pipermail/python-3000/2007-April/007063.html .. [#lexical] (email subject) PEP-3125 -- remove backslash continuation, Koenig http://mail.python.org/pipermail/python-3000/2007-May/007237.html .. [#snocone] The Snocone Programming Language, Koenig http://www.snobol4.com/report.htm .. [#guidobughide] (email subject) PEP-3125 -- remove backslash continuation, van Rossum http://mail.python.org/pipermail/python-3000/2007-May/007244.html Copyright ========= This document has been placed in the public domain. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End:

1 0

Re: [Python-Dev] [Python-3000] Pre-pre PEP for 'super' keyword
by Delaney, Timothy (Tim) 04 May '07

04 May '07

Guido van Rossum wrote: >> 1. When a method is defined, the class is bound to it via an attribute >> (which in my version is called func_class). > In Py3k all the func_XXX attrs are renamed __XXX__, so this would be > __class__; but that's a name reserved for something else, so it would > need to be something else. E.g. __containing_class__. Yep - I've just used a placeholder name. > Also, this would have to be done when the class is defined; when the > method is being defined the class doesn't exist yet. Good point. Change that to "at the first opportunity" ;) >> 2. Every non-static method has an implicit cell variable called >> 'super'. > > I think you're using 'cell' in a different sense than it is normally > used in Python's implementation. What you are looking for is called a > local variable (I deduce this from your initialization of the "cell" > with something computed from the first argument). Actually, I think I'm using the terminology correctly - I'm talking about an entry in co_cellvars. Given the following class: class A(object): def f(self): super = super_factory() def inner(): return 'A' + super.f() print inner() the use of 'super' in both A.f and A.f.inner will produce an entry in A.f.func_code.co_cellvars and A.f.inner.func_code.co_freevars. What I'm proposing is that the `super = super_factory()` line be implicit in this case, resulting in the following code behaving identically: class A(object): def f(self): def inner(): return 'A' + super.f() print inner() >> The issue of super() vs. super.__call__() ambiguity - I'll need to >> look at that when I get home. > > Sounds irrelevant -- if super is equivalent to > __builtin__.__super__(<class>, <firstarg>) then super never gets > called; the only thing ever done to it (in the normal course of > things) is to request an attribute of it. Sorry - this is related to my proposal that the following two bits of code behave the same: class A(object): def f(self, *p, **kw): super.f(*p, **kw) class A(object): def f(self, *p, **kw): super(*p, **kw) But as has been pointed out, this creates an ambiguity with: class A(object): def f(self, *p, **kw): super.__call__(*p, **kw) so I want to see if I can resolve it. > super(ThisClass).method(...) # ??? > > The third exists so that you can create an "unbound" super instance > which is useful for the oft-misunderstood autosuper example in my > "descrintro" essay: > http://www.python.org/download/releases/2.2.3/descrintro/#metaclass_exam ples > . > > But since the latter is the hack that we're trying to replace with a > proper implementation here, I suspect we can get away with only > supporting the first two forms (and the third form is still supported > using __builtin__.__super__). Yep - that's my thought as well. I think it would be very rare to need super(ThisClass), although it makes some sense that that would be what super means in a static method ... Tim Delaney

6 7

Re: [Python-Dev] [Python-3000] Pre-pre PEP for 'super' keyword
by Jim Jewett 04 May '07

04 May '07

On 4/29/07, Tim Delaney <tcdelaney(a)optusnet.com.au> wrote: > I've been intending to write up a PEP for fixing super, but I haven't had > time to get to it. Calvin Spealman has the most recent draft. I hope he will incorporate this into his draft. > 1. 'super' becomes a keyword, that returns a super object for the instance > method currently being executed. So it is a "keyword" in the sense that None is a keyword; not in the stronger sense that "if" is a keyword? > 4. super objects are callable, and calling them will execute the super > method with the same name as the instance method currently being executed. > Lookup of this method occurs when the instance method is entered. > > class A(object): > def f(self): > pass > > class B(A): > def f(self): > super() # Calls A.f(self) > If you want name lookup to occur at the time of the call, you can explicitly > specify the method name (just like with any other super attribute): > > class A(object): > def f(self): > pass > > class B(A): > def f(self): > super.f() # Calls A.f(self) As long as you can be explicit, should the shortcut be a full shortcut? That is, def f(self, a, b=c, *args, **kwargs): super() # passes the exact arglist that f got vs def __init__(self, myvar, passed_var): super.__init__(self, passed_var) # flags that you are changing the args -jJ

5 7

2.6, x64 and PGO
by Kristján Valur Jónsson 03 May '07

03 May '07

Well, I have checked in the fixes to the trunk to make an x64 build run the testsuite. It works pretty well. I did a quick test and found the x64Release to run 51000 pystones vs. 44000 for the win32Release version, a difference of some 10%. And the x64PGO version ran some 61000 pystones. Some more extensive testing is of course required, but it would seem that gains can be had from both going 64 bit and using PGO. I encourage you to take a look at the PGO build process. It is much simpler than before. I also think we ought to find out if we could start producing a proper vc8 build, setting up a buildbot, etc. I can probably convince management at CCP to arrange for one. We could also think about a way to have the 64 bit and 32 bit versions coexist on the same machine. Cheers, Kristjan

2 1

Re: [Python-Dev] PEP 30XZ: Simplified Parsing
by Raymond Hettinger 03 May '07

03 May '07

[Skip] > I use it all the time. For example, to build up (what I consider to be) >readable SQL queries: > > rows = self.executesql("select cities.city, state, country" > " from cities, venues, events, addresses" > " where cities.city like %s" > " and events.active = 1" > " and venues.address = addresses.id" > " and addresses.city = cities.id" > " and events.venue = venues.id", > (city,)) I find that style hard to maintain. What is the advantage over multi-line strings? rows = self.executesql(''' select cities.city, state, country from cities, venues, events, addresses where cities.city like %s and events.active = 1 and venues.address = addresses.id and addresses.city = cities.id and events.venue = venues.id ''', (city,)) Raymond

5 7

64 bit warnings
by Kristján Valur Jónsson 02 May '07

02 May '07

There is a considerable amount of warnings present for 64 bit builds on windows. You can see them using VisualStudio 2005 even if you don't have the x64 compilers installed, by turning on "Detect 64 bit portability issues" in the general tab for pythoncore. Now, some of those just need straightforward upgrades of loop counters and so on to Py_ssize_t. Others probably require more judgement. E.g., do we want to change the signature of PyEval_EvalCodeEx() to accept Py_ssize_t counters rather than int? And if not, should we then use Py_SAFE_DOWNCAST() or just regular (int) typecast? Note that on x64 there is rarely any performance cost associated with usin 64 bit variables for function calls, since most of the time arguments are passed in registers. i.e. it is mostly structs that we want to keep unchanged, imo. Any thoughts? Kristján

2 1

head crashing (was: Fwd: [Python-checkins] buildbot warnings in x86 mvlgcc trunk)
by Neal Norwitz 02 May '07

02 May '07

This is the third time I've seen a crash on 2 different machines. This is the first time I noticed this unexplained crash: http://python.org/dev/buildbot/all/amd64%20gentoo%20trunk/builds/1983/step-… That was at r54982. I tried to reproduce this: with a non-debug build, with a debug build, with valgrind with both types of build. I could never reproduce it. Valgrind did not report any errors either. Here is the third failure: http://python.org/dev/buildbot/all/amd64%20gentoo%20trunk/builds/1986/step-… The failure below prints: python: Objects/obmalloc.c:746: PyObject_Malloc: Assertion `bp != ((void *)0)' failed. which probably doesn't really help since the corruption has already occurred. See http://python.org/dev/buildbot/all/x86%20mvlgcc%20trunk/builds/497/step-tes… Anyone have ideas what might have caused this? n -- ---------- Forwarded message ---------- From: buildbot(a)python.org <buildbot(a)python.org> Date: Apr 30, 2007 11:17 PM Subject: [Python-checkins] buildbot warnings in x86 mvlgcc trunk To: python-checkins(a)python.org The Buildbot has detected a new failure of x86 mvlgcc trunk. Full details are available at: http://www.python.org/dev/buildbot/all/x86%2520mvlgcc%2520trunk/builds/497 Buildbot URL: http://www.python.org/dev/buildbot/all/ Build Reason: Build Source Stamp: [branch trunk] HEAD Blamelist: georg.brandl Build had warnings: warnings test Excerpt from the test logfile: make: *** [buildbottest] Aborted (core dumped) sincerely, -The Buildbot _______________________________________________ Python-checkins mailing list Python-checkins(a)python.org http://mail.python.org/mailman/listinfo/python-checkins

7 9

Re: [Python-Dev] New operations in Decimal
by Nick Maclaren 01 May '07

01 May '07

Facundo Batista <facundo(a)taniquetil.com.ar> wrote: > > > Am I losing my marbles, or is this a proposal to add the logical > > operations to FLOATING-POINT? > > Sort of. This is a proposal to keep compliant with the General Decimal > Arithmetic Specification, as we promised. > > http://www2.hursley.ibm.com/decimal/ Or, more precisely: http://www2.hursley.ibm.com/decimal/damisc.html All right. Neither you nor I have lost our marbles, but the authors of that assuredly did. It's totally insane. And implementing it for a software emulation of that specification built on top of a twos complement binary integer model is insanity squared. But promises are promises and mere insanity is not in itself an obstacle to political success .... I shall attempt to forget that I ever asked the question :-) Regards, Nick Maclaren, University of Cambridge Computing Service, New Museums Site, Pembroke Street, Cambridge CB2 3QH, England. Email: nmm1(a)cam.ac.uk Tel.: +44 1223 334761 Fax: +44 1223 334679

1 0