Mailman 3 May 2001 - Python-Dev

Off-topic: So long, and thanks for all the fish
by Guido van Rossum 20 May '01

20 May '01

For all you Douglas Adams fans out there: Douglas Noel Adams 1952 - 2001 http://www.douglasadams.com --Guido van Rossum (home page: http://www.python.org/~guido/)

1 0

[off topic] Python is taking over the world
by James C. Ahlstrom 20 May '01

20 May '01

I was in my local (Sommerville, NJ) Borders book store last week and noticed that they stocked many Python books, most in multiple copies. It all added up to three feet of Python books. Great. The clincher was when I went to my YMCA, and saw that someone had posted a flyer offering tutoring in Math, Physics, Java and Python. Congratulations to Guido and all on this list. JimA

1 0

PyBench DictCreation (was Re: Performance compares)
by Tim Peters 19 May '01

19 May '01

The worst percentage hit in both MAL's and Jeremy's pybench run was (here showing Jeremy's numbers, cuz I doubt anyone could reproduce MAL's <wink>): DictCreation: 87.80 ms 2.93 us +115.72% Assorted things do not account for it: the new overhead of linking and unlinking dicts into the gc list (at creation and destruction times) seems to account for no more than 2%; and the overhead due to using the slower lookdict (as opposed to lookdict_string) even less. Jeremy cheated by running a profiler: the true cause is that dictresize gets called about twice as often. Before 2.1: *before* inserting an item, we checked to see whether the dict was at the resize point. If so, we resized it. Note that this meant PyDict_SetItem could grow a dict even if no new entry was made (and that this was the cause of several excruciating bugs in the 2.1 release cycle, since it meant a dict could get reshuffled merely when replacing the values associated with existing keys). 2.1: *after* inserting an item, and if the key was new (i.e., the dict grew a new entry, as opposed to just replacing the value associated with an existing key), and the dict is at the resize point, we resize it. Now the DictCreation test overwhelmingly creates dicts of size exactly 3. The dict resizes from empty to capacity 4 on the way to gaining 2 entries. When adding the third: Before 2.1: 2 < (2/3)*4 == 2 2/3, so the dict is not resized and ends up remaining a capacity-4 dict with 3 slots full. This actually violates a documented dict invariant (i.e., that dicts are never more than 2/3rd full). 2.1: The third item added is a new item, and 3 > (2/3)*4 == 2 2/3, so we *do* resize it, and the dict ends up with 3 of 8 slots full. I've got no interest in trying to restore the old behavior. A compromise may be to boost the minimum size of a non-empty dict from 4 to 8. As is, the only non-empty dicts that can get away with using the current minimum size of 4 have no more than 2 elements. The question is whether such tiny non-empty dicts are common enough to make everyone else pay for "an extra" resize. go-ahead-just-*try*-to-prove-your-answer<wink>-ly y'rs - tim

8 19

Re: [Python-checkins] CVS: python/dist/src/Lib HTMLParser.py,NONE,1.1
by Fredrik Lundh 18 May '01

18 May '01

guido wrote: > A much improved HTML parser -- a replacement for sgmllib. The API is > derived from but not quite compatible with that of sgmllib, so it's a > new file. I suppose it needs documentation, and htmllib needs to be > changed to use this instead of sgmllib, and sgmllib needs to be > declared obsolete. any reason this cannot be made compatible with sgmllib? Cheers /F

2 1

IPv6
by Martin v. Loewis 18 May '01

18 May '01

> What's out IPv6 story? I recall that someone once sent me patches, > but they didn't work for me. Is it time to try again? In certain > circles IPv6 support in Python would be enough to switch programming > languages... :-) It's still on SF, http://sourceforge.net/tracker/index.php?func=detail&aid=401196&group_id=54… There are two problems with that patch, AFAICT: 1. It is too large for any individual to review in one chunk. 2. It gets quickly outdated. 3. It touches core aspects of the socket handling that are IMO better untouched. I don't know whether the generalization proposed there is necessary to support IPv6 reasonably - the author certainly feels it is. To integrate the patch, I would propose to split it into smaller parts, and submit and review them one-by-one. The first patch should deal only with autoconf stuff, so that the proper #defines are in config.h (although they would not be used right away). The second patch should be a tar file of all new files (the patch on SF actually misses some files). The third patch should include changes to the C modules, and the last one changes to the standard library modules. For that procedure to work, we need cooperation from the submitter. For that, we probably need to indicate that we are really interested in his work, and will work with him to integrate it into Python. So far, his impression must be that nobody is interested - the patch is sitting there since 2000-08-16, making it the oldes open patch. Undoubtedly, integrating this piece of work will result in various problems with Python CVS: it won't build anymore on "funny machines" (like Windows), and it might even crash on code that used to work just fine. This prediction is not based on the actual content of the patch, merely on its size, and the fact that IPv6 support is experimental on many systems. So we'ld also need a BDFL pronouncement that we really really want this, and that anybody running into problems should either help fixing them, or stay away from CVS while it is being integrated. Regards, Martin

1 0

Re: [Python-Dev] RE: [Python-checkins] CVS: python/dist/src/Modules spam.c,1.1.2.3,1.1.2.4
by Uche Ogbuji 18 May '01

18 May '01

> [Guido] > > Index: spam.c > > ... > > Congratulations! "My other" ISP (MSN) just started tagging suspected spam > with "spam" in the subject line, and my mail reader moves that to a special > spam folder upon delivery. So far this is the one and only incoming email > it's moved. Many solicitations to help foreign nationals move large sums of > money out of their country have gotten through [...] I thought I was th only one getting all these silly Nigerian scam spams. I figured maybe they saw my name and decided to test on me (though they might more cleverly have figured that a fellow Nigerian would be wise to the game). However, with the (sloppily) bogus headers I've always found on those things, I'm surprised your ISP couldn't sniff them out. Not that it matters. The Eastern Nigerian proverb gets it right. "Once hunters learn to shoot without missing, birds will learn to fly without resting". -- Uche Ogbuji Principal Consultant uche.ogbuji(a)fourthought.com +1 303 583 9900 x 101 Fourthought, Inc. http://Fourthought.com 4735 East Walnut St, Ste. C, Boulder, CO 80301-2537, USA Software-engineering, knowledge-management, XML, CORBA, Linux, Python

4 3

GC and ExtensionClass - a summary of the problem and a workaround
by skip＠pobox.com 17 May '01

17 May '01

Over the past couple days I've included python-dev on various messages in an ongoing thread about a segmentation violation I was getting with the new PyGtk2 wrappers. With some excellent assistance from the GC maestro, Neil Schemenauer, I finally know what's going on and I have a simple workaround that lets me get back to work. Here's a summary of the problem. When defining ExtensionClass types, you need to create and initialize a PyExtensionClass struct. It looks something like so: PyExtensionClass PyGtkTreeSortable_Type = { PyObject_HEAD_INIT(NULL) 0, /* ob_size */ "GtkTreeSortable", /* tp_name */ sizeof(PyPureMixinObject), /* tp_basicsize */ ... }; Note that the parameter to the PyObject_HEAD_INIT macro is NULL. It would normally be the address of a type object (e.g. &PyType_Type). However, Jim Fulton pointed out that on Windows you can't get the address of &PyType_Type object at compile time. Accordingly, ExtensionClass provides a PyExtensionClass_Export macro whose responsibility is, in part, to set the ob_type field appropriately at runtime. (I'm not sure why this Windows nit doesn't afflict other type declarations like PyTuple_Type. I'm sure others will know why. I just accept Jim's word as gospel and move on...) A problem arises if the garbage collector runs while the module initialization function is running, but before all the ob_type fields have been assigned their correct values. In this case, a one-element tuple representing the bases of a particular PyGtk extension class was traversed by the garbage collector. The workaround turns out to be exceedingly simple: import gc gc.disable() import gtk gc.enable() I can handle doing that from Python code for the time being and will leave it up to others to decide how, if at all, ExtensionClass should be changed to correct the problem. Skip

7 12

Re: [Python-Dev] Performance compares
by Michael Hudson 17 May '01

17 May '01

"M.-A. Lemburg" <mal(a)lemburg.com> writes: > Sure are... but I'm not so much interested in the absolute numbers > -- it's the hot-spots which showed up that scare me: e.g. dictionary > creation seems to have suffered along the way for some reason, > functions calls are even slower now than they were previously and > other important tasks such a instance creation take a similar hit > (probably as a result of the other two). Have you tried fiddling with gc parameters? If the GC does a multi generation trawl through the heap in the middle of some test, that might skew the numbers in unexpected ways. Or not, of course. Cheers, M. -- CLiki pages can be edited by anybody at any time. Imagine the most fearsomely comprehensive legal disclaimer you have ever seen, and double it -- http://ww.telent.net/cliki/index

2 2

Re: [Python-Dev] Easy codec access
by Michael Hudson 17 May '01

17 May '01

Guido van Rossum <guido(a)digicool.com> writes: > > I've just checked in a set of patches which implement the new > > .decode() method along with a couple of useful codecs. > > Cool! Indeed. Good idea, Marc! This is a bit unfriendly though: >>> "bobbins".encode("gzip") Traceback (most recent call last): File "<stdin>", line 1, in ? File "/usr/local/src/python/dist/build/Lib/encodings/__init__.py", line 59, in search_function raise SystemError,\ SystemError: module "encodings.gzip" failed to register I thought SystemErrors shouldn't ever happen (isn't it what gets raised for an illegal opcode, for example?). > > To see just how easy it is to write codecs, please have > > a look at the string codecs I added in this patch (e.g. > > zlib_codec.py or hex_codec.py). I am pretty sure that there > > are a lot more useful things in the standard lib which could > > benefit from these easy-to-use interfaces. > > As an excercise, I added a quoted-printable codec. It was easy > indeed! urlencode would be nice. Maybe re.escape, too. html entities? That's probably a bigger can of worms, but print "<p>%s</p>"%text.encode("html") seems delightfully simpleminded. Cheers, M. -- GAG: I think this is perfectly normal behaviour for a Vogon. ... VOGON: That is exactly what you always say. GAG: Well, I think that is probably perfectly normal behaviour for a psychiatrist. -- The Hitch-Hikers Guide to the Galaxy, Episode 9

5 6

FYI: MIT's dynamic language design panel now online
by Chris Ryland 17 May '01

17 May '01

This talk is most entertaining! Highly recommended to you good folk, if only as a reinforcement of the good design principles embodied in Python (with the exception of print >> ;-). Jonathan Rees (an old Scheme/T hand) kept referring to Python whenever he wanted to give an example of a modern dynamic language (disclaiming a lot of knowledge about it). He mentioned it three or four times (usually positively), so it must be on the tip of his mind. -- Cheers! Chris Ryland Em Software, Inc. www.emsoftware.com

1 0