Mailman 3 November 2005 - Python-Dev

Re: [Python-Dev] Plea to distribute debugging lib
by David Abrahams 05 Nov '05

05 Nov '05

Bronek Kozicki <brok(a)rubikon.pl> writes: > David Abrahams wrote: >> Who knows what the whiny babies will accept? That said, I think >> people would be happy with a .zip file containing whatever is built by >> selecting the debug build in the VS project and asking it to build >> everything. (**) > > Just to clarify - what we are asking for is library built with _DEBUG > and no BOOST_DEBUG_PYTHON, that is the one compatible with default > Python distribution. Bronek, I know you're trying to help, but I'm sure that's not making anything clearer for these people. They don't know anything about BOOST_DEBUG_PYTHON and would never have cause to define it. -- Dave Abrahams Boost Consulting www.boost-consulting.com

2 2

Problems with revision 4077 of new SVN repository
by Robin Munn 04 Nov '05

04 Nov '05

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I'm trying to mirror the brand-new Python SVN repository with SVK, to better be able to track both the trunk and the various branches. Since I'm not a Python developer and don't have svn+ssh access, I'm doing so over http. The process fails when trying to fetch revision 4077, with the following error message: "RA layer request failed: REPORT request failed on 'projects/!svn/bc/41373/python': The REPORT request returned invalid XML in the response: XML parse error at line 7: not well-formed (invalid token) (/projects/!svn/bc/41373/python)" The thread at http://svn.haxx.se/dev/archive-2004-07/0793.shtml suggests that the problem may lie in the commit message for revision 4077: if it has a character in the 0x01-0x1f range (which are invalid XML), then Subversion methods like http: will fail to retrieve it, while methods like file: will succeed. I haven't tried svn+ssh: since I don't have an SSH key on the server. Trying "svn log -r 4077 http://svn.python.org/projects/python/" also fails: subversion/libsvn_ra_dav/util.c:780: (apr_err=175002) svn: REPORT request failed on '/projects/!svn/bc/4077/python' subversion/libsvn_ra_dav/util.c:760: (apr_err=175002) svn: The REPORT request returned invalid XML in the response: XML parse error at line 7: not well-formed (invalid token) (/projects/!svn/bc/4077/python) When I visit http://svn.python.org/view/python/?rev=4077, I can see the offending log message. Sure enough, there's a 0x1b character in it, between the space after "Added" and the "h" immediately before the word "Moved". This problem can be fixed by someone with root permissions on the SVN server logging in and running the following: echo "New commit message goes here" > new-message.txt svnadmin setlog --bypass-hooks -r 4077 /path/to/repos new-message.txt If there are other, similar problems later in the SVN repository, I was unable to find them because the SVK mirror process consistently halts at revision 4077. If revision 4077 is fixed and I turn up other log problems, I'll report them as well. - -- Robin Munn rmunn(a)pobox.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.0 (Darwin) Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org iD8DBQFDaRd46OLMk9ZJcBQRApjAAJ9K3Y5z1q4TulqwVjmZTZb9ZgY31ACcD8RI fNFmGL2U4XaIKa2n6UUyxEA= =tEbq -----END PGP SIGNATURE-----

4 9

Adding examples to PEP 263
by Fredrik Lundh 04 Nov '05

04 Nov '05

the runtime warning you get when you use non-ascii characters in python source code points the poor user to this page: http://www.python.org/peps/pep-0263.html which tells the user to add a # -*- coding: <encoding name> -*- to the source, and then provides a more detailed syntax description as a RE pattern. to help people that didn't grow up with emacs, and don't speak fluent RE, and/or prefer to skim documentation, it would be a quite helpful if the page also contained a few examples; e.g. # -*- coding: utf-8 -*- # -*- coding: iso-8859-1 -*- can anyone with SVN write access perhaps add this? (I'd probably add a note to the top of the page for anyone who arrives there via a Python error message, which summarizes the pep and provides an example or two; abstracts and rationales are nice, but if you're just a plain user, a "do this; here's how it works; further discussion below" style is a bit more practical...) </F>

3 2

Proposal: can we have a python-dev-announce mailing list?
by Jack Jansen 04 Nov '05

04 Nov '05

As people may have noticed (or possibly not:-) I've been rather inactive on python-dev the last year or so, due to being completely inundated with other work. Too bad that I've missed all the interesting discussions on Python 3000, but I'm bound to catch up some time later this year:-). BUT: what I also missed are all the important announcements, such as new releases, the switch to svn, and a couple more (I think). I know I would be much helped with a moderated python-dev-announce mailing list, which would be only low-volume, time-critical announcements for people developing Python. Even during times when I am actively following python-dev it would be handy to have important announcements coming in in a separate mailbox in stead of buried under design discussions and such... -- Jack Jansen, <Jack.Jansen(a)cwi.nl>, http://www.cwi.nl/~jack If I can't dance I don't want to be part of your revolution -- Emma Goldman

6 6

PYTHOPN_API_VERSION
by Thomas Heller 04 Nov '05

04 Nov '05

Shouldn't PYTHON_API_VERSION be different between 2.3 and 2.4? It is 1012 in both versions. I tried to detect whether PyTuple_Pack is supported, which was added in 2.4. Or is this only to detect changed apis, and not added apis? Thomas

2 1

PyPy 0.8.0 is released!
by Michael Hudson 03 Nov '05

03 Nov '05

pypy-0.8.0: Translatable compiler/parser and some more speed ============================================================== The PyPy development team has been busy working and we've now packaged our latest improvements, completed work and new experiments as version 0.8.0, our third public release. The highlights of this third release of PyPy are: - Translatable parser and AST compiler. PyPy now integrates its own compiler based on Python own 'compiler' package but with a number of fixes and code simplifications in order to get it translated with the rest of PyPy. This makes using the translated pypy interactively much more pleasant, as compilation is considerably faster than in 0.7.0. - Some Speed enhancements. Translated PyPy is now about 10 times faster than 0.7 but still 10-20 times slower than CPython on pystones and other benchmarks. At the same time, language compliancy has been slightly increased compared to 0.7 which had already reached major CPython compliancy goals. - Some experimental features are now translateable. Since 0.6.0, PyPy shipped with an experimental Object Space (the part of PyPy implementing Python object operations and manipulation) implementing lazily computed objects, the "Thunk" object space. With 0.8.0 this object space can also be translated preserving its feature additions. What is PyPy (about)? ------------------------------------------------ PyPy is a MIT-licensed research-oriented reimplementation of Python written in Python itself, flexible and easy to experiment with. It translates itself to lower level languages. Our goals are to target a large variety of platforms, small and large, by providing a compilation toolsuite that can produce custom Python versions. Platform, Memory and Threading models are to become aspects of the translation process - as opposed to encoding low level details into a language implementation itself. Eventually, dynamic optimization techniques - implemented as another translation aspect - should become robust against language changes. Note that PyPy is mainly a research and development project and does not by itself focus on getting a production-ready Python implementation although we do hope and expect it to become a viable contender in that area sometime next year. PyPy is partially funded as a research project under the European Union's IST programme. Where to start? ----------------------------- Getting started: http://codespeak.net/pypy/dist/pypy/doc/getting-started.html PyPy Documentation: http://codespeak.net/pypy/dist/pypy/doc/ PyPy Homepage: http://codespeak.net/pypy/ The interpreter and object model implementations shipped with the 0.8 version can run on their own and implement the core language features of Python as of CPython 2.4. However, we still do not recommend using PyPy for anything else than for education, playing or research purposes. Ongoing work and near term goals --------------------------------- At the last sprint in Paris we started exploring the new directions of our work, in terms of extending and optimising PyPy further. We started to scratch the surface of Just-In-Time compiler related work, which we still expect will be the major source of our future speed improvements and some successful amount of work has been done on the support needed for stackless-like features. This release also includes the snapshots in preliminary or embryonic form of the following interesting but yet not completed sub projects: - The OOtyper, a RTyper variation for higher-level backends (Squeak, ...) - A JavaScript backend - A limited (PPC) assembler backend (this related to the JIT) - some bits for a socket module PyPy has been developed during approximately 16 coding sprints across Europe and the US. It continues to be a very dynamically and incrementally evolving project with many of these one-week workshops to follow. PyPy has been a community effort from the start and it would not have got that far without the coding and feedback support from numerous people. Please feel free to give feedback and raise questions. contact points: http://codespeak.net/pypy/dist/pypy/doc/contact.html have fun, the pypy team, (Armin Rigo, Samuele Pedroni, Holger Krekel, Christian Tismer, Carl Friedrich Bolz, Michael Hudson, and many others: http://codespeak.net/pypy/dist/pypy/doc/contributor.html) PyPy development and activities happen as an open source project and with the support of a consortium partially funded by a two year European Union IST research grant. The full partners of that consortium are: Heinrich-Heine University (Germany), AB Strakt (Sweden) merlinux GmbH (Germany), tismerysoft GmbH (Germany) Logilab Paris (France), DFKI GmbH (Germany) ChangeMaker (Sweden), Impara (Germany)

1 0

Re: [Python-Dev] Should the default equality operator compare valuesinstead of identities?
by Noam Raphael 03 Nov '05

03 Nov '05

On 11/2/05, Raymond Hettinger <raymond.hettinger(a)verizon.net> wrote: > > Should the default equality operator compare valuesinstead of > identities? > > No. Look back into last year's python-dev postings where we agreed that > identity would always imply equality. There were a number of practical > reasons. Also, there are a number of places in CPython where that > assumption is implicit. > Perhaps you've meant something else, or I didn't understand? Identity implying equality is true also in value-based comparison. If the default __eq__ operator compares by value, I would say that it would do something like: def __eq__(self, other): if self is other: return True if type(self) is not type(other): return False (compare the __dict__ and any __slots__, and if they are all ==, return True.) Noam

1 0

Optimizations on the AST representation
by Rune Holm 03 Nov '05

03 Nov '05

Hi, I'm a norwegian applied mathematics student with an interest in compilers, and I've been a long-time python user and a python-dev lurker for some time. I'm very happy that you've integrated the AST branch into mainline, but I noticed that the AST compiler does not perform much optimization yet, so I though I'd take a crack at it. I just submitted the following patches: http://www.python.org/sf/1346214 http://www.python.org/sf/1346238 which adds better dead code elimination and constant folding of the AST representation to Python. The constant folding patch adds two new files, Include/optimize.h and Python/optimize.c, which includes a general visitor interface abstracted from exisiting visitor code for easy optimization pass creation. The code is in new files in order to make room for more AST optimization passes, and since Python/compile.c is already quite crowded with byte code generation and bytecode optimization. If desired, this patch could changed to add code to Python/compile.c instead. Further work: A limited form of type interference (e.g. as a synthesized attribute) could be very useful for further optimizations. Since python allows operator overloading, it isn't safe to perform strength reductions on expressions with operands of unknown type, as there is no way to know if algebraic identities will hold. However, if we can infer from the context that expressions have the type of int, float or long, many optimizations become possible, for instance: x**2 => x*x x*2 => x+x x*0 => 0 x*1 => x 4*x + 5*x => 9*x (this optimization actually requires common subexpression elimination for the general case, but simple cases can be performed without this) and so forth. Another interesting optimization that can potensially bring a lot of additional speed is hoisting of loop invariants, since calling python methods involves allocating and creating a method-wrapper object. An informal test shows that optimizing lst = [] for i in range(10): lst.append(i+1) into lst = [] tmp = lst.append for i in range(10): tmp(i+1) will yield a 10% speed increase. This operation is of course not safe with arbitrary types, but with the above type interference, we could perform this operation if the object is of a type that disallows attribute assignment, for instance lists, tuples, strings and unicode strings. Regards, Rune Holm

1 0

Re: [Python-Dev] apparent ruminations on mutable immutables (was:PEP 351, the freeze protocol)
by Michael Chermside 03 Nov '05

03 Nov '05

Josiah Carlson writes: > If you make such a suggestion, I would offer that you create a new PEP, > because this discussion has gone beyond PEP 351, and has wandered into > the realm of "What other kinds of objects would be interesting to have > in a Python-like system?" Noam Raphael replies: > That is a good suggestion, and I have already started to write one. It > takes me a long time, but I hope I will manage. My thanks to both of you... following this conversation has been an educational experience. Just for the record, I wanted to chime in with my own opinion formed after following the full interchange. I think Noam's propsal is very interesting. I like the idea of allowing both "frozen" (ie, immutable) and mutable treatments for the same object. I think that C++'s version of this concept (the "const" modifier) has, on balance, been only a very limited success. I find myself convinced by Noam's claims that many common use patterns either (1) only use mutables, or (2) only use immutables, or (3) only use immutable copies temporarily and avoid mutating while doing so. Any such use patterns (particularly use (3)) would benefit from the presence of an efficient method for creating an immutable copy of a mutable object which avoids the copy where possible. However... it seems to me that what is being described here is not Python. Python is a wonderful language, but it has certain characteristics, like extremely dynamic behavior and close integration with underlying system methods (C in CPython, Java in Jython, etc) that seem to me to make this particular feature a poor fit. That's OK... not all languages need to be Python! I would encourage you (Noam) to go ahead and explore this idea of yours. You might wind up building a new language from scratch (in which case I strongly encourage you to borrow _syntax_ from Python -- its syntax is more usable than that of any other language I know of). Or perhaps you will prefer to take CPython and make minor modifications. This kind of experimentation is allowed (open source) and even encouraged... consider Christian Tismer's Stackless -- a widely admired variant of CPython which is unlikely to ever become part of the core, but is nevertheless an important part of the vivrant Python community. You might even be interested in starting, instead, with PyPy -- an large project which has as its main goal producing an implementation of Python which is easy to modify so as to support just this kind of experimentation. You are also welcome to submit a PEP for modifying Python (presumably CPython, Jython, Iron Python, and all other implementations). However, I think such a PEP would be rejected. Building your own thing that works well with Python would NOT be rejected. The idea is interesting, and it _may_ be sound; only an actual implementation could prove this either way. -- Michael Chermside

2 1

Re: [Python-Dev] Why should the default hash(x) == id(x)?
by Michael Chermside 02 Nov '05

02 Nov '05

Noam Raphael writes: > Is there a reason why the default __hash__ method returns the id of the objects? > > It is consistent with the default __eq__ behaviour, which is the same > as "is", but: > > 1. It can easily become inconsistent, if someone implements __eq__ and > doesn't implement __hash__. > 2. It is confusing: even if someone doesn't implement __eq__, he may > see that it is suitable as a key to a dict, and expect it to be found > by other objects with the same "value". > 3. If someone does want to associate values with objects, he can > explicitly use id: > dct[id(x)] = 3. This seems to better explain what he wants. Your first criticism is valid... it's too bad that there isn't a magical __hash__ function that automatically derived its behavior from __eq__. To your second point, I would tell this user to read the requirements. And your third point isn't a criticism, just an alternative. But to answer your question, the reason that the default __hash__ returns the ID in CPython is just that this works. In Jython, I belive that the VM provides a native hash method, and __hash__ uses that instead of returning ID. Actually, it not only works, it's also FAST (which is important... many algorithms prefer that __hash__ being O(1)). I can't imagine what you would propose instead. Keep in mind that the requirements are that __hash__ must return a value which distinguishes the object. So, for instance, two mutable objects with identical values MUST (probably) return different __hash__ values as they are distinct objects. > This leads me to another question: why should the default __eq__ > method be the same as "is"? Another excellent question. The answer is that this is the desired behavior of the language. Two user-defined object references are considered equal if and only if (1) they are two references to the same object, or (2) the user who designed it has specified a way to compare objects (implemented __eq__) and it returns a True value. > Why not make the default __eq__ really compare the objects, that is, > their dicts and their slot-members? Short answer: not the desired behavior. Longer answer: there are three common patterns in object design. There are "value" objects, which should be considered equal if all fields are equal. There are "identity" objects which are considered equal only when they are the same object. And then there are (somewhat less common) "value" objects in which a few fields don't count -- they may be used for caching a pre-computed result for example. The default __eq__ behavior has to cater to one of these -- clearly either "value" objects or "identity" objects. Guido chose to cater to "identity" objects believing that they are actually more common in most situations. A beneficial side-effect is that the default behavior of __eq__ is QUITE simple to explain, and if the implementation is easy to explain then it may be a good idea. -- Michael Chermside

1 0