Mailman 3 May 2001 - Python-Dev

python-dev summary 2001-05-10 - 2001-05-24
by Michael Hudson May 24, 2001

May 24, 2001

This is a summary of traffic on the python-dev mailing list between May 10 and May 24 (inclusive) 2001. It is intended to inform the wider Python community of ongoing developments. To comment, just post to python-list(a)python.org or comp.lang.python in the usual way. Give your posting a meaningful subject line, and if it's about a PEP, include the PEP number (e.g. Subject: PEP 201 - Lockstep iteration) All python-dev members are interested in seeing ideas discussed by the community, … [View More]so don't hesitate to take a stance on a PEP if you have an opinion. This is the eighth summary written by Michael Hudson. Summaries are archived at: <http://starship.python.net/crew/mwh/summaries/> Posting distribution (with apologies to mbm) Number of articles in summary: 322 | [|] | [|] 30 | [|] | [|] [|] [|] [|] | [|] [|] [|] [|] | [|] [|] [|] [|] [|] | [|] [|] [|] [|] [|] | [|] [|] [|] [|] [|] [|] [|] 20 | [|] [|] [|] [|] [|] [|] [|] [|] | [|] [|] [|] [|] [|] [|] [|] [|] [|] | [|] [|] [|] [|] [|] [|] [|] [|] [|] [|] [|] | [|] [|] [|] [|] [|] [|] [|] [|] [|] [|] [|] [|] | [|] [|] [|] [|] [|] [|] [|] [|] [|] [|] [|] [|] [|] | [|] [|] [|] [|] [|] [|] [|] [|] [|] [|] [|] [|] [|] 10 | [|] [|] [|] [|] [|] [|] [|] [|] [|] [|] [|] [|] [|] | [|] [|] [|] [|] [|] [|] [|] [|] [|] [|] [|] [|] [|] | [|] [|] [|] [|] [|] [|] [|] [|] [|] [|] [|] [|] [|] | [|] [|] [|] [|] [|] [|] [|] [|] [|] [|] [|] [|] [|] | [|] [|] [|] [|] [|] [|] [|] [|] [|] [|] [|] [|] [|] | [|] [|] [|] [|] [|] [|] [|] [|] [|] [|] [|] [|] [|] [|] 0 +-023-025-017-018-028-031-036-032-025-002-015-018-020-032 Thu 10| Sat 12| Mon 14| Wed 16| Fri 18| Sun 20| Tue 22| Fri 11 Sun 13 Tue 15 Thu 17 Sat 19 Mon 21 Wed 23 Pretty busy fortnight. The above distribution may be somewhat skewed because I changed my subscription address to python-dev and was unsubscribed for a while. Although any impact this had is probably countered by ESR and Barry's discussion of "Puffy the Frog"... * Type/class * Paul Prescod has been keeping an eye on Guido's descr-branch work, and posted concerns about when objects will have a __dict__: <http://mail.python.org/pipermail/python-dev/2001-May/014694.html> Then there was more technical discussion about subclassing builtin types and Steven Majewski evangelising prototype-based OO languages (though I'm not sure why!). * Easy codec access * Marc-Andre Lemburg checked in his decode string method patch, and some new codecs so you can now do things like: >>> "abc".encode('zlib').encode('base64') 'eJxLTEoGAAJNASc=\n' >>> _.decode('base64').decode('zlib') 'abc' There was a small discussion on what other codecs might be handy and Guido added quoted-printable to check it was easy. * Performance * The big discussion(s) on python-dev over the past fourteen days has centred on performance, especially on that of comparisons and the related area of dict performance. It all started with Tim Peters running a simple test program on 2.0, 2.1 and current CVS: <http://mail.python.org/pipermail/python-dev/2001-May/014781.html> The discussion had an unusual <wink> flavour for one about performance: a concentration on measuring performance numbers and making sure that the optimizations being discussed actually improved these numbers. This is hard; everyone wants to speed the "typical Python app" but of course there is no such thing; people have been using, amongst others, pystone, pybench and the test suite, none of which are particularly good candidates... Tim posted the distribution of sizes of dicts in a run of the test suite: <http://mail.python.org/pipermail/python-dev/2001-May/014890.html> which showed that small dicts are overwhelmingly the commonest. Marc piped up with an old optimization idea of his: <http://mail.python.org/pipermail/python-dev/2001-May/014891.html> He posted a patch to sourceforge, Tim rewrote it and checked it in, so dicts should be a little faster in 2.2. But as I said, the discussion was kicked off by the performance of comparisons, especially strings. Martin von Loewis posted some statistics from an instrumented interpreter: <http://mail.python.org/pipermail/python-dev/2001-May/014808.html> The issue is that the rich comparisons of Python 2.1 have added a layer of complexity to the comparisons code. Although the rich comparisons (might) provide an opportunity for faster code in some circumstances, code that still uses old-style comparisons can and does take a hit. Strings still use the old-style comparisons and are compared a *lot* (especially in dicts), so it seems "upgrading" them to rich comparisons should be a win and Marc posted a patch to sf that does this. Marc also managed to promise <wink> to make a concerted effort to find speed optimizations in the next few months: <http://mail.python.org/pipermail/python-dev/2001-May/014928.html> Finally, in a coda Jeremy noticed that Python spends an alarming amount of time decoding those "Oi|s#" strings that get passed to PyArg_ParseTuple: <http://mail.python.org/pipermail/python-dev/2001-May/014911.html> and Tim pointed out that optimizing "O" might be a win: <http://mail.python.org/pipermail/python-dev/2001-May/014924.html> * FP vs. tutorial * Tim pointed out that the tutorial currently contains examples of floating point output that is platform dependent, and that this is bad. He proposed changing the tutorial to only use fractions that can be exactly represented as floats, and adding a discussion (possibly in an appendix) of the reasons why >>> 0.1 0.10000000000000001 is not broken. There was a discussion of how detailed the discussion should be where the point was made that it's not really important to explain precisely *why* this happens, but it suffices to convince the newbie that floating point is more complicated than he or she thinks. Lets hope that suitable text is composed soon, and that people actually read it ... there have been two "floating point is broken" bug reports on sourceforge in just the last week. * unifying os.rename semantics across platforms * Skip pointed out that os.rename behaves differently on Posix and Windows platforms when the destination file exists: <http://mail.python.org/pipermail/python-dev/2001-May/014957.html> on Posix the destination is silently replaced in an atomic operation, whereas on Windows an exception is raised. Skip proposed enforcing posix semantics everywhere, but this has two problems (a) it's backwards incompatible (b) it's impossible (you can't avoid the race condition on Windows). So maybe we'll just settle for better documentation. * Python 2.1.1 * Thomas Wouters started back-porting bug fixes to the 2,1-maint branch in preparation for a 2.1.1 release. There is as yet no firm - or even vague - plans about release dates. * Daily Python-URL on your Palm * Marc-Andre Lemburg announced that you can now read Pythonware's Daily Python-URL on your Palm Pilot as an AvantGo channel: <http://mail.python.org/pipermail/python-dev/2001-May/014983.html> Cheers, M. [View Less]

1 0

Re: [Python-Dev] Python 2.1.1
by Thomas Wouters May 24, 2001

May 24, 2001

[ Answer CC'd to python-dev since it deserves an official answer :) ] On Thu, May 24, 2001 at 08:37:17AM +0100, Michael Hudson wrote: > For summarasing purposes, do you have any idea when Python 2.1.1 will > be released? > "No" is a perfectly acceptable answer. Then "No" it is ! Even though I have a fair bit of patches in the queue right now, I need some more time to check out (no pun intended) the changes since the fork, and I want to browse the bug list for possible bugs that … [View More]

1 0

pre-PEP on optimized global names
by Jeremy Hylton May 23, 2001

May 23, 2001

I've been hoping to work on optimized global and builtin name support for Python 2.2. I'm not sure if I'll have time, but thought I'd circulate a draft with some notes on the subject now. Anyone interested in this work? Jeremy PEP: ??? Title: Optimized Access to Module and Builtin Names Author: jeremy(a)digicool.com (Jeremy Hylton) Status: Draft Type: Standards Track Python-Version: 2.2 Created: 23-May-2001 Abstract This PEP proposes a new implementation of global module namespaces … [View More] and the builtin namespace that speeds name resolution. The implementation would use an array of object pointers for most operations in these namespaces. The compiler would assign indices for global variables at compile time. The current implementation represents these namespaces as dictionaries. A global name incurs a dictionary lookup each time it is used; a builtin name incurs two dictionary lookups, a failed lookup in the global namespace and a second lookup in the builtin namespace. This implementation should speed Python code that uses module-level functions and variables. It should also eliminate awkward coding styles that have evolved to speed access to these names. The implementation is complicated because the global and builtin namespaces can be modified dynamically in ways that are impossible for the compiler to detect. (Example: A module's namespace is modified by a script after the module is imported.) As a result, the implementation must maintain several auxillary data structures to preserve these dynamic features. Introduction [expand on the basic ideas in the abstract] [describe the key parts of the design: dlict, compiler support, stupid name trick workarounds, optimization of other module's globals] DLict design The namespaces are implemented using a data structure that has sometimes gone under the name dlict. It is a dictionary that has numbered slots for some dictionary entries. The type must be implemented in C to achieve acceptable performance. A Python implementation is included here to explain the basic design: """A dictionary-list hybrid""" import types class DLict: def __init__(self, names): assert isinstance(names, types.DictType) self.names = {} self.list = [None] * size self.empty = [1] * size self.dict = {} self.size = 0 def __getitem__(self, name): i = self.names.get(name) if i is None: return self.dict[name] if self.empty[i] is not None: raise KeyError, name return self.list[i] def __setitem__(self, name, val): i = self.names.get(name) if i is None: self.dict[name] = val else: self.empty[i] = None self.list[i] = val self.size += 1 def __delitem__(self, name): i = self.names.get(name) if i is None: del self.dict[name] else: if self.empty[i] is not None: raise KeyError, name self.empty[i] = 1 self.list[i] = None self.size -= 1 def keys(self): if self.dict: return self.names.keys() + self.dict.keys() else: return self.names.keys() def values(self): if self.dict: return self.names.values() + self.dict.values() else: return self.names.values() def items(self): if self.dict: return self.names.items() else: return self.names.items() + self.dict.items() def __len__(self): return self.size + len(self.dict) def __cmp__(self, dlict): c = cmp(self.names, dlict.names) if c != 0: return c c = cmp(self.size, dlict.size) if c != 0: return c for i in range(len(self.names)): c = cmp(self.empty[i], dlict.empty[i]) if c != 0: return c if self.empty[i] is None: c = cmp(self.list[i], dlict.empty[i]) if c != 0: return c return cmp(self.dict, dlict.dict) def clear(self): self.dict.clear() for i in range(len(self.names)): if self.empty[i] is None: self.empty[i] = 1 self.list[i] = None def update(self): pass def load(self, index): """dlict-special method to support indexed access""" if self.empty[index] is None: return self.list[index] else: raise KeyError, index # XXX might want reverse mapping def store(self, index, val): """dlict-special method to support indexed access""" self.empty[index] = None self.list[index] = val def delete(self, index): """dlict-special method to support indexed access""" self.empty[index] = 1 self.list[index] = None Compiler issues The compiler currently collects the names of all global variables in a module. These are names bound at the module level or bound in a class or function body that declares them to be global. The compiler would assign indices for each global name and add the names and indices of the globals to the module's code object. Each code object would then be bound irrevocably to the module it was defined in. (Not sure if there are some subtle problems with this.) Enhancement: Optimized access to other module's globals If one module imports another and binds a name in the global namespace, the compiler currently detects that the particular global is bound to a module. The compiler also note access to any attribute of a module, and emit special opcodes for accessing these names. At runtime the implementation can lookup the index of the module attribute in the module's namespace. In the current namespace, a pointer to the foreign module's dlict can be recorded along with the name's offset in the dlict. This would allow names, e.g. types.StringType, to be used with the same efficiency as globals. Backwards compatibility The dlict will need to maintain metainformation about whether a slot is currently used or not. It will also need to maintain a pointer to the builtin namespace. When a name is not currently used in the global namespace, the lookup will have to fail over to the builtin namespace. In the reverse case, each module may need a special accessor function for the builtin namespace that checks to see if a global shadowing the builtin has been added dynamically. This check would only occur if there was a dynamic change to the module's dlict, i.e. when a name is bound that wasn't discovered at compile-time. These mechanisms would have little if any cost for the common case whether a module's global namespace is not modified in strange ways at runtime. They would add overhead for modules that did unusual things with global names, but this is an uncommon practice and probably one worth discouraging. It may be desirable to disable dynamic additions to the global namespace in some future version of Python. If so, the new implementation could provide warnings. Local Variables: mode: indented-text indent-tabs-mode: nil End: [View Less]

1 0

Import hook to do end-of-line conversion?
by Jack Jansen May 23, 2001

May 23, 2001

[Oops, try again] There's talk on the PythonMac-SIG to create an import hook that would read modules with either \r, \n or \r\n newlines and convert them to the local convention before feeding them to the rest of the import machinery. The reason this has become interesting is the mixed unixness/macness of MacOSX, where such an import hook could be used to share a Python tree between MacPython and bsd-Python. They would only need a different site.py (probably), living somehwere near the head of … [View More]

15 56

Assertion failed in dictobject.c
by Jack Jansen May 23, 2001

May 23, 2001

I'm seeing the assert on line 525 in dictobject.c (revision 2.92) failing. The debugger tells me that ma_fill and ma_size are both 8. ma_used is 2, and interestingly hash is also 8. Going back to revision 2.90 fixes the problem (or masks it). -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen(a)oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.oratrix.nl/~jack | ++++ see http://www.xs4all.nl/~tank/ ++++

2 2

Re: Simulating Class (was Re: Does Python have Class methods)
by James_Althoff＠i2.com May 23, 2001

May 23, 2001

Python-dev'ers, Pardon the intrusion, but Aahz Maruch suggested that I post this to the python-dev list. The message below illustrates "yet another class method recipe" that Costas synthesized (and which I then modified very slightly) from various posts following another discussion on python-list about class methods (as we all await the "type/class healing" stuff some of you are working on -- go team!). This variant uses explicit "metaclasses" (defined as regular classes) whose instances ("… [View More]meta objects") point to class objects (since they cannot *be* class objects in current Python). Anyway, I think the approach has some nice properties. Best regards, Jim ----- Forwarded by James Althoff/AMER/i2Tech on 05/18/01 11:23 AM ----- James Althoff To: python-list(a)python.org 05/14/01 02:09 cc: PM Subject: Re: Simulating Class (was Re: Does Python have Class methods)(Document link: James Althoff) Costas writes: >Ok, so after looking thru how Python works and comments from people, I >came up with what I believe may be the best way to implement Class >methods and Class variables. > ><snip> > >Costas I think this idea is quite good. I would amend it very slightly by suggesting the convention of defining *three* separate names in the enclosing module: 1) the name of the enclosing class 2) the name of the singleton instance of the enclosing class 3) the name of the enclosed class To support this, I would propose using a naming convention as below. If one is interested in defining a class Spam, then use the following names: 1) SpamMetaClass -- names the enclosing class 2) SpamMeta -- names a singleton instance of the enclosing class 3) Spam -- names the enclosed class Use the name SpamMetaClass when you need to derive a subclass of SpamMetaClass, e.g., class SpecialSpamMetaClass(SpamMetaClass): pass Use the name SpamMeta to invoke a class method, e.g., SpamMeta.aClassMethod() Use the name Spam to make instances as usual, e.g., s = Spam() (and to derive a subclass of Spam). Although SpamMetaClass is not a metaclass in the sense of Smalltalk or Ruby -- that is to say, the class Spam is not an instance of SpamMetaClass -- nonetheless, SpamMetaClass still acts as a "higher level" class that provides methods on behalf of the class Spam where said methods are 1) independent of any particular instance of Spam and 2) allow for factory-method-style creation of Spam instances -- these being two very important attributes of the metaclass concept. Plus "meta" is a nice, short name. :-) Plus using "MetaClass" to refer to the class and "Meta" to refer to the singleton instance of "MetaClass" is reasonably clear and succinct, I think. One nice thing about the proposed recipe is that the SpamMeta object is a real class instance of a real class. This means that -- unlike when using the "module function" recipe -- we get inheritance of methods, and -- unlike when using the "callable wrapper class" recipe -- we also get override of methods. The example below illustrates both of these important capabilities. class Class1MetaClass: # Base metaclass # Define "class methods" for Class1 def whoami(self): print 'Class1MetaClass.whoami:', self def new(self): # Factory method """Return a new instance""" return self.Class1() def newList(self,n=3): # Another factory method """Return a list of new instances""" l = [] for i in range(n): newInstance = self.new() l.append(newInstance) return l # Define Class1 & its "instance methods" class Class1: # Base class def whoami(self): print 'Class1.whoami:', self Class1Meta = Class1MetaClass() # Make & name the singleton metaclass instance Class1 = Class1Meta.Class1 # Make the Class1 name accessible class Class2MetaClass(Class1MetaClass): # Derived metaclass # Define "class methods" for Class2 -- Override Class1 "class methods" def whoami(self): print 'Class2MetaClass.whoami:', self def new(self): # Override the factory method return self.Class2() # Define Class2 & its "instance methods" class Class2(Class1): # Derived class def whoami(self): print 'Class2.whoami:', self Class2Meta = Class2MetaClass() # Make & name the singleton metaclass instance Class2 = Class2Meta.Class2 # Make the Class2 name accessible # Test Class1Meta.whoami() # invoke "class method" of base class Class2Meta.whoami() # invoke "class method" of derived class Class1().whoami() # make an instance & invoke "instance method" Class2().whoami() print Class1Meta.newList() # factory method print Class2Meta.newList() # inherit factory method with override >>> reload(meta6) Class1MetaClass.whoami: <meta6.Class1MetaClass instance at 00810DBC> Class2MetaClass.whoami: <meta6.Class2MetaClass instance at 00812D6C> Class1.whoami: <meta6.Class1 instance at 0081058C> Class2.whoami: <meta6.Class2 instance at 0081058C> [<meta6.Class1 instance at 0081147C>, <meta6.Class1 instance at 0081151C>, <meta6.Class1 instance at 0081009C>] [<meta6.Class2 instance at 0081147C>, <meta6.Class2 instance at 00810CCC>, <meta6.Class2 instance at 0081009C>] <module 'meta6' from 'c:\_dev\python20\meta6.py'> Jim [View Less]

3 3

Daily Python URL on your Palm
by M.-A. Lemburg May 23, 2001

May 23, 2001

Just thought you might want to know that Fredrik's Daily Python URL can be downloaded onto the Palm as Avantgo Channel. Here's the URL for adding the channel: http://avantgo.com/mydevice/autoadd.html?title=Daily%20Python%20URL&url=htt… PS: Would be nice if Fredrik could provide a "printable" version of the Daily URL page, since the table layout doesn't work too well on the small Palm display. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH … [View More]

1 0

unifying os.rename semantics across platform
by skip＠pobox.com May 23, 2001

May 23, 2001

Couldn't figure out why this message never generated any comment. Turns out it didn't reach the list because the host I sent it from (dynamic4.tttech.com) couldn't be resolved. I just noticed it in my errors mailbox and am sending it out again. ------------------------------------------------------------------------------ It was brought to my attention a week ago by a client that os.rename semantics differ between Unix and Windows. On Unix, if the destination file already exists it is … [View More]

6 8

Re: [Python-Dev] FP vs. tutorial
by Tim Peters May 23, 2001

May 23, 2001

Let's get some errors cleared up first: + FixedPoint is not in the distribution. + There is no PEP for Decimal. + Decimal f.p. is not more accurate than binary f.p. In fact, it's provably worse (but not by much). For the rest, + Yes, I'm serious about not including tutorial examples with platform-dependent output, unless they're explicitly meant to illustrate non-portable code. + Specific small examples notwithstanding, there is no uniformity across platforms in the last digit or … [View More]

7 17

Re: [Python-Dev] FP vs. tutorial
by Michael Hudson May 22, 2001

May 22, 2001

Michel Pelletier <michel(a)digicool.com> writes: > Who was it that said every equation will halve your audience? It was Stephen Hawking's editor when he was preparing A Brief History Of Time (or at least, it gets mentioned in the preface; the advice may be older). Cheers, M. -- 7. It is easier to write an incorrect program than understand a correct one. -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html

2 1