Mailman 3 May 2006 - Python-Dev

Changing python int to "long long".
by Sean Reifschneider 24 May '06

24 May '06

We've been discussing the possibility of converting the Python int type to long long (from long). I played around with it some, and it's going to be a huge change that probably will break most C extensions until. However, as uncletimmy says, "Python is so stinking slow" that it probably won't make much of a negative impact on the performance. The reason we're looking at this at Need for Speed is that EWT has a lot of code that uses ints between the 32 and 64 bit range. It'd be a hard change to make to convert to long long for regular ints. I'm going to speak with John and Runar today about if we can make just a long long extension type and live with it not integrating with other type coercion as well as an int would. The big deal right now is on 32 bit platforms, giving the 64-bits for int. However, it will also be a win for 64-bit platforms for ints that fall between 64 and 128 bits. So, I ran pybench on a half-arsed conversion of ints to long long, and only lost around 1.11% over all. However, for simple math I got a solid 25% speed-up (n = ((((i << 33) * 2) + 1) / 3) - 2), and for 1<<35 I got a 34% speed up, compared with them getting converted to Python longs. For ints that will fit entirely within 32-bits, I'm seeing around 11% worse performance when going to 64-bit types. This is all on the 2.5 trunk. My conclusion is that it's probably enough of a performance drop that we can't just do it wholesale. It has a big gain for ints within 32 and 64 on 32-bit platforms, but for <32-bit ints it's going to be pretty bad. Within the next few years many platforms, particularly performance-critical ones, will probably be 64 bit anyway, so fewer people will probably be able to see an advantage to it. The ideal would probably be to have another int type and up-convert to a long long, THEN go to arbitrary precision, but that's a pretty big change. Less big than converting the int to long long though. The easiest would be to make an extension that has long long, if that will work for speeding up applications like EWT has. I'll follow up with them on this. Any thoughts on this? Thanks, Sean -- Linux: When you need to run like a greased weasel. -- Sean Reifschneider, 1998 Sean Reifschneider, Member of Technical Staff <jafo(a)tummy.com> tummy.com, ltd. - Linux Consulting since 1995: Ask me about High Availability

4 3

Re: [Python-Dev] [Python-checkins] r46064 - in python/trunk: Include/Python.h Include/pyport.h Misc/ACKS Misc/NEWS Modules/_localemodule.c Modules/main.c Modules/posixmodule.c Modules/sha512module.c PC/pyconfig.h Python/thread_nt.h
by Neal Norwitz 24 May '06

24 May '06

On 5/22/06, martin.v.loewis <python-checkins(a)python.org> wrote: > Author: martin.v.loewis > Date: Mon May 22 11:15:18 2006 > New Revision: 46064 > > Modified: python/trunk/Include/Python.h > ============================================================================== > --- python/trunk/Include/Python.h (original) > +++ python/trunk/Include/Python.h Mon May 22 11:15:18 2006 > @@ -35,7 +35,9 @@ > #endif > > #include <string.h> > +#ifndef DONT_HAVE_ERRNO_H > #include <errno.h> > +#endif > #include <stdlib.h> > #ifdef HAVE_UNISTD_H > #include <unistd.h> What is the reason for the DONT_HAVE_* macros? Can we use the HAVE_* versions? There only seem to be 22 occurrences total: Include/pyport.h:6 Modules/arraymodule.c:2 Modules/getbuildinfo.c:1 Modules/selectmodule.c:1 Objects/fileobject.c:3 PC/pyconfig.h:2 Python/ceval.c:1 Python/mystrtoul.c:1 Python/strtod.c:1 Python/thread.c:1 RISCOS/pyconfig.h:3 n

3 3

SSH key for work computer
by Brett Cannon 24 May '06

24 May '06

Can someone install the attached SSH key (it's for my work machine)? The fingerprint is:: cd:69:15:52:b2:e5:dc:2e:73:f1:62:1a:12:49:2b:a1 bcannon(a)google.com Also, how hard is it to have a specific key uninstalled? The reason I ask is that my internship is only for three months so that after that the key won't be valid. -Brett

2 1

2.5 schedule
by Jim Jewett 24 May '06

24 May '06

Thomas Wouters wrote: > The fact that much testing goes on between releases > as well makes the alphas even less important. I suspect the amount of MS Windows testing done between releases is fairly small. And what little there is doesn't always use the same compiler that will eventually be used for the final release. A snapshot build would fill this gap almost as well as a formal alpha. -jJ

1 0

Re: [Python-Dev] Decimal and Exponentiation
by Tim Peters 23 May '06

23 May '06

[elventear] > I am the in the need to do some numerical calculations that involve > real numbers that are larger than what the native float can handle. > > I've tried to use Decimal, but I've found one main obstacle that I > don't know how to sort. I need to do exponentiation with real > exponents, but it seems that Decimal does not support non integer > exponents. > > I would appreciate if anyone could recommend a solution for this > problem. Wait <0.3 wink>. Python's Decimal module intends to be a faithful implementation of IBM's proposed standard for decimal arithmetic: http://www2.hursley.ibm.com/decimal/ Last December, ln, log10, exp, and exponentiation to non-integral powers were added to the proposed standard, but nobody yet has written implementation code for Python's module. [Python-Dev: somebody wants to volunteer for this :-)] If you're not a numeric expert, I wouldn't recommend that you try this yourself (in particular, trying to implement x**y as exp(ln(x)*y) using the same precision is mathematically correct but is numerically badly naive). The GNU GMP library (for which Python bindings are available) also supports "big floats", but their power operation is also restricted to integer powers and/or exact roots. This can be painful even to try; e.g., >>> from gmpy import mpf >>> mpf("1e10000") ** mpf("3.01") consumed well over a minute of CPU time (on a 3.4 GHz box) before dying with ValueError: mpq.pow fractional exponent, inexact-root If you're working with floats outside the range of IEEE double, you _probably_ want to be working with logarithms instead anyway; but that depends on you app, and I don't want to know about it ;-)

2 2

Looking for Memories of Python Old-Timers
by Guido van Rossum 23 May '06

23 May '06

How long have you used Python? 10 years or longer? Please tell us how you first heard of the language, how you first used it, and how you helped develop it (if you did). More recent reminiscences are welcome too! Please add them to my blog: http://www.artima.com/weblogs/viewpost.jsp?thread=161207 -- --Guido van Rossum (home page: http://www.python.org/~guido/)

3 2

Re: [Python-Dev] [Web-SIG] Adding wsgiref to stdlib
by Phillip J. Eby 23 May '06

23 May '06

It's not clear to me whether this means that Ian can just relicense his code for me to slap into wsgiref and thence into Python by virtue of my own PSF contribution form and the compatible license, or whether it means Ian has to sign a form too. At 09:25 PM 5/22/2006 -0700, Guido van Rossum wrote: >This explains what to do, and which license to use: > >http://www.python.org/psf/contrib/ > >--Guido > >On 5/22/06, Ian Bicking <ianb(a)colorstudy.com> wrote: >>Phillip J. Eby wrote: >> > At 02:32 PM 4/28/2006 -0500, Ian Bicking wrote: >> >> I'd like to include paste.lint with that as well (as wsgiref.lint or >> >> whatever). Since the last discussion I enumerated in the docstring all >> >> the checks it does. There's still some outstanding issues, mostly where >> >> I'm not sure if it is too restrictive (marked with @@ in the source). >> >> It's at: >> >> >> >> http://svn.pythonpaste.org/Paste/trunk/paste/lint.py >> > >> > Ian, I see this is under the MIT license. Do you also have a PSF >> > contributor agreement (to license under AFL/ASF)? If not, can you place >> > a copy of this under a compatible license so that I can add this to the >> > version of wsgiref that gets checked into the stdlib? >> >>I don't have a contributor agreement. I can change the license in >>place, or sign an agreement, or whatever; someone should just tell me >>what to do. >> >> >>-- >>Ian Bicking | ianb(a)colorstudy.com | http://blog.ianbicking.org >>_______________________________________________ >>Python-Dev mailing list >>Python-Dev(a)python.org >>http://mail.python.org/mailman/listinfo/python-dev >>Unsubscribe: >>http://mail.python.org/mailman/options/python-dev/guido%40python.org > > >-- >--Guido van Rossum (home page: http://www.python.org/~guido/)

2 1

PEP-xxx: Unification of for statement and list-comp syntax
by Heiko Wundram 23 May '06

23 May '06

Hi all! The following PEP tries to make the case for a slight unification of for statement and list comprehension syntax. Comments appreciated, including on the sample implementation. === PEP: xxx Title: Unification of for-statement and list-comprehension syntax Version: $Revision$ Last-Modified: $Date$ Author: Heiko Wundram <me(a)modelnine.org> Status: Active Type: Standards Track Content-Type: text/plain Created: 21-May-2006 Post-History: 21-May-2006 17:00 GMT+0200 Abstract When list comprehensions were introduced, they added the ability to add conditions which are tested before the expression which is associated with the list comprehension is evaluated. This is often used to create new lists which consist only of those items of the original list which match the specified condition(s). For example: [node for node in tree if node.haschildren()] will create a new list which only contains those items of the original list (tree) whose items match the havechildren() condition. Generator expressions work similarily. With a standard for-loop, this corresponds to adding a continue statement testing for the negated expression at the beginning of the loop body. As I've noticed that I find myself typing the latter quite often in code I write, it would only be sensible to add the corresponding syntax for the for statement: for node in tree if node.haschildren(): <do something with node> as syntactic sugar for: for node in tree: if not node.haschildren(): continue <do something with node> There are several other methods (including generator-expressions or list-comprehensions, the itertools module, or the builtin filter function) to achieve this same goal, but all of them make the code longer and harder to understand and/or require more memory, because of the generation of an intermediate list. Implementation details The implementation of this feature requires changes to the Python grammar, to allow for a variable number of 'if'-expressions before the colon of a 'for'-statement: for_stmt: 'for' exprlist 'in' testlist_safe ('if' old_test)* ':' suite ['else' ':' suite] This change would replace testlist with testlist_safe as the 'in'-expression of a for statement, in line with the definition of list comprehensions in the Python grammar. Each of the 'if'-expressions is evaluated in turn (if present), until one is found False, in which case the 'for'-statement restarts at the next item from the generator of the 'in'-expression immediately (the tests are thus short-circuting), or until all are found to be True (or there are no tests), in which case the suite body is executed. The behaviour of the 'else'-suite is unchanged. The intermediate code that is generated is modelled after the byte-code that is generated for list comprehensions: def f(): for x in range(10) if x == 1: print x would generate: 2 0 SETUP_LOOP 42 (to 45) 3 LOAD_GLOBAL 0 (range) 6 LOAD_CONST 1 (10) 9 CALL_FUNCTION 1 12 GET_ITER >> 13 FOR_ITER 28 (to 44) 16 STORE_FAST 0 (x) 19 LOAD_FAST 0 (x) 22 LOAD_CONST 2 (1) 25 COMPARE_OP 2 (==) 28 JUMP_IF_FALSE 9 (to 40) 31 POP_TOP 3 32 LOAD_FAST 0 (x) 35 PRINT_ITEM 36 PRINT_NEWLINE 37 JUMP_ABSOLUTE 13 >> 40 POP_TOP 41 JUMP_ABSOLUTE 13 >> 44 POP_BLOCK >> 45 LOAD_CONST 0 (None) 48 RETURN_VALUE where all tests are inserted immediately at the beginning of the loop body, and jump to a new block if found to be false which pops the comparision from the stack and jumps back to the beginning of the loop to fetch the next item. Implementation issues The changes are backwards-compatible, as they don't change the default behaviour of the 'for'-loop. Also, as the changes that this PEP proposes don't change the byte-code structure of the interpreter, old byte-code continues to run on Python with this addition unchanged. Implementation A sample implementation (with updates to the grammar documentation and a small test case) is available at: http://sourceforge.net/tracker/index.php?func=detail&aid=1492509&group_id=5… Copyright This document has been placed in the public domain. === --- Heiko.

8 21

Re: [Python-Dev] PEP-xxx: Unification of for statement andlist-comp syntax
by Delaney, Timothy (Tim) 23 May '06

23 May '06

Niko Matsakis wrote: > For what it's worth, I also favor the continue syntax Heiko compared > his code against. I also commonly do early tests and either break or continue, but my criteria is whether it makes the flow easier to understand. Sometimes it does, sometimes it doesn't. (on two lines of course ;) where things are considerably more complex > In general, I favor the idea that the main flow of > computation should have the least indentation. Ditto, simply because it tends to be more readable. > I also am +1 on this > PEP, even if it is doomed, as I've often longed to use list- > comprehension like syntax in a for loop; to me it is mentally taxing > to remember which syntax is supported where. > > Just thought I'd throw Heiko some support. :) I'm -1 because there are numerous other ways to do it that are just as clear IMO. No support from me Heiko ;) Tim Delaney

1 0

Re: [Python-Dev] [Python-checkins] r46043 - peps/trunk/pep-0356.txt
by Delaney, Timothy (Tim) 23 May '06

23 May '06

Thomas Wouters wrote: > On 5/22/06, skip(a)pobox.com <skip(a)pobox.com> wrote: > > Tim> If there's no functionality changes, what would be the > problem with Tim> putting it in post-alpha? > > It still represents new code that may introduce new bugs. In theory > (and > generally in practice for Python), once you move into the beta stage > all you > do is fix bugs. > > > Note that the problem isn't 'post-alpha', it's 'post-beta' -- new > features (especially minor or long-agreed-upon) are fine up until > beta1 ;) Isn't "beta" post-alpha? <wink> I did of course mean once we were out of the alpha stage. If they're purely performance changes (no new functionality) then there's no problem with putting it in during beta - IMO even if it introduces new bugs (so long as those bugs are gone by the time 2.5 goes final). Beta is "feature set frozen". Tim Delaney

1 0